Accelerated Gradient-CQ Algorithms for Split Feasibility Problems

Abstract

This work focuses on split feasibility problems in Hilbert spaces. To accelerate the convergent rate of gradient-CQ algorithms, we introduce an inertial term. Additionally, non-monotone stepsizes are employed to adjust the relaxation parameter applied to the original stepsizes, ensuring that these original stepsizes maintain a positive lower bound. Thereby, the efficiency of the algorithms is improved. Moreover, the weak and strong convergence of the proposed algorithms are established through proofs that exhibit a similar symmetry structure and do not require the assumption of Lipschitz continuity for the gradient mappings. Finally, the LASSO problem is presented to illustrate and compare the performance of the algorithms.

1. Introduction

This work focuses on the split feasibility problem ($SFP$), as follows

$$\begin{array}{c}\begin{array}{c}\mathrm{Seek}{S}^{*}\in C\mathrm{such} \mathrm{that}A{S}^{*}\in Q.\hfill \end{array}\hfill \end{array}$$

where $H_1$ and $H_2$ are both real Hilbert spaces, the closed convex sets $C\subset H_1$ and $Q\subset H_2$ are nonempty, and $A:H_1\to H_2$ is a bounded linear operator with adjoint $A^{*}$. If a point $S^{*}$ exists, then the solution set of $SFP$ is given by

$$\Gamma =\{S^{*}\in C:A{S}^{*}\in Q\}.$$

Censor and Elfving [1] was the first to study the original formulation of $SFP$ for modeling inverse problems. This model has since been widely applied to practical problems such as signal processing [2] and medical image reconstruction [3]. Because of its wide range of applications, numerous numerical algorithms have been proposed to solve the $SFP$; see [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18] and the references therein. Among these, a classical method for solving the $SFP$ is Byrne’s CQ algorithm [2,3], which us defined by the following scheme:

$${\vartheta }_{n+1}=P_C\left({\vartheta }_n-\tau A^{*}(I-P_Q)A{\vartheta }_n\right),\forall n\ge 1,$$

(1)

where $\tau \in \left(0,{\displaystyle \frac{2}{{\parallel A\parallel }^2}}\right)$, while $P_C$ and $P_Q$ are projections onto C and Q, respectively. In numerical experiments, sets C and Q are defined as follows

$$\begin{array}{c}C=\left\{{\zeta }_1\in H_1:c\left({\zeta }_1\right)\le 0\right\},Q=\left\{{\zeta }_2\in H_2:q\left({\zeta }_2\right)\le 0\right\},\hfill \end{array}$$

(2)

where $c:H_1\to \mathbb{R}$ and $q:H_2\to \mathbb{R}$ are two proper convex functions. Since the projections $P_C$ and $P_Q$ do not have closed-form expressions, algorithm (1) is not applicable. To address this, Yang [19] proposed the following weakly convergent algorithm:

$${\vartheta }_{n+1}=P_{C_n}\left({\vartheta }_n-{\tau }_n\nabla V_n\left({\vartheta }_n\right)\right),$$

(3)

where the stepsize is ${\tau }_n\in \left(0,{\displaystyle \frac{2}{{\parallel A\parallel }^2}}\right)$ and

$$C_n=\{{\zeta }_1\in H_1:c\left({\vartheta }_n\right)\le ⟨{\epsilon }_n,{\vartheta }_n-{\zeta }_1⟩\}$$

with ${\epsilon }_n\in \partial c\left({\vartheta }_n\right)$ and

$$Q_n=\{{\zeta }_2\in H_2:q\left(A{\vartheta }_n\right)\le ⟨{\zeta }_n,A{\vartheta }_n-{\zeta }_2⟩\}$$

with ${\zeta }_n\in \partial q\left(A{\vartheta }_n\right)$, and the function $V_n\left({\zeta }_1\right)$ with a gradient of $\nabla V_n\left({\zeta }_1\right)$ is defined in Lemma 6. Evidently, $C\subset C_n$ and $Q\subset Q_n$, and the projections onto the half-spaces $C_n$ and $Q_n$ have closed-form expressions.Therefore, algorithm (3) performs well. However, the fixed step size ${\tau }_n\in \left(0,{\displaystyle \frac{2}{{\parallel A\parallel }^2}}\right)$ is quite conservative, which negatively impacts the numerical performance of Byrne’s CQ algorithm. To overcome this limittion, many self-updated step size algorithms have been proposed to solve the $SFP$; see, e.g., [7,8,10,11,12,13,14,16,17,19,20,21,22,23,24]. Among them, Kesornprom et al. [22] proposed the following gradient-CQ algorithm for solving the $SFP$

$$\begin{array}{c}{z}_n={\vartheta }_n-{\tau }_n\nabla V_n\left({\vartheta }_n\right)\hfill \\ {\vartheta }_{n+1}=P_{C_n}(z_n-{\lambda }_n\nabla V_n\left(z_n\right)),\hfill \end{array}$$

(4)

where ${\tau }_n={\displaystyle \frac{{\gamma }_n{V}_n\left({\vartheta }_n\right)}{\parallel \nabla V_n\left({\vartheta }_n\right){\parallel }^2+{\theta }_n}}$ and ${\lambda }_n={\displaystyle \frac{{\gamma }_n{V}_n\left(z_n\right)}{\parallel \nabla V_n\left(z_n\right){\parallel }^2+{\theta }_n}}$, $0<{\gamma }_n<4$, $0<{\theta }_n<1$. They proved the weak convergence of algorithm (4) under the conditions $\underset{n}{inf}{\gamma }_n(4-{\gamma }_n)>0$ and $\underset{n\to \infty }{lim}{\theta }_n=0.$ To obtain strong convergence, they presented the following gradient-CQ algorithm through Halpern’s iteration process [25,26,27]. Specifically, the fixed vector $u\in H_1$ is provided and the initial guess ${\vartheta }_1$ is chosen arbitrarily. The sequence $\left\{{\vartheta }_n\right\}$ is computed as follows:

$$\begin{array}{c}{z}_n={\vartheta }_n-{\tau }_n\nabla V_n\left({\vartheta }_n\right)\hfill \\ {\vartheta }_{n+1}={\pi }_{n}u+(1-{\pi }_n)P_{C_n}(z_n-{\lambda }_n\nabla V_n\left(z_n\right)),\hfill \end{array}$$

(5)

where ${\tau }_n$ and ${\lambda }_n$ are given in (4). The resulting sequence $\left\{{\vartheta }_n\right\}$ converges strongly to $P_{\Gamma }u$ under the conditions $\underset{n}{inf}{\gamma }_n(4-{\gamma }_n)>0$, $\underset{n\to \infty }{lim}{\theta }_n=0$, $\left\{{\pi }_n\right\}\subset (0,1)$, ${\displaystyle \underset{n\to \infty }{lim}}{\pi }_n=0\mathrm{and}{\displaystyle \sum _{n=1}^{\infty }}{\pi }_n=\infty .$

To accelerate the convergent rate of relaxed CQ algorithms, the inertial extrapolation technique [28] is adopted, and inertial relaxed CQ algorithms have been introduced; see, e.g., [5,6,8,9,10,11]. Recently, Ma and Liu [11] proved the strong convergence of the relaxed CQ approach [9] using the following scheme.

$$\left\{\begin{array}{c}{R}_n={\vartheta }_n+{\Re }_n({\vartheta }_n-{\vartheta }_{n-1}),z_n=P_{C_n}(R_n-{\tau }_n\nabla V_n\left(R_n\right)),\hfill \\ {\vartheta }_{n+1}={\pi }_{n}u+(1-{\pi }_n)z_n,\hfill \\ {\tau }_{n+1}=\left\{\begin{array}{cc}\min \left\{{\displaystyle \frac{2\delta V_n\left(R_n\right)}{\parallel \nabla V_n\left(R_n\right){\parallel }^2}},d_n{\tau }_n+{\psi }_n\right\},\hfill & \mathrm{if}\nabla V_n\left(R_n\right)\ne 0,\hfill \\ d_n{\tau }_n+{\psi }_n,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\hfill \end{array}\right.$$

(6)

where ${\Re }_n$ is the inertial term, $d_n$ and ${\psi }_n$ are found through Lemma 3.

Note that under the assumptions of Lipschitz continuity for the gradient operator and firm nonexpansiveness of $I-P_{Q_n}$, the described algorithms exhibit certain features. In algorithms (4) and (5), the relaxation parameters ${\theta }_n$ and $\underset{n}{inf}{\gamma }_n(4-{\gamma }_n)>0$ are applied directly to the step sizes.As a result, the step sizes lack positive lower bounds, possible leading to conservative choices and, consequently, slow convergence, as observed in [29]. The convergence of these algorithms is established under the aforementioned conditions. However, the study of gradient-CQ algorithms with inertial effects, such as in (6), has not yet been established.This raises a natural question:

Question: Can we develop new modifications of algorithms (4) and (5) that not only improve numerical performance but also ensure convergence by relaxing the assumptions of Lipschitz continuity for the gradient operator $\nabla V_n$ and firm nonexpansiveness for the mapping $I-P_{Q_n}$?

In response to the proposed question, we answer in the affirmative. The main contributions of this work are as follows:

-

We include the inertial idea [8,9,11,28,30,31] into algorithms (4) and (5) to accelerate their convergence rate and establish new convergence results.

-

We adopt double non-Lipschitz step sizes [9], which remove the restrictions imposed by the condition $\underset{n}{inf}{\gamma }_n(4-{\gamma }_n)>0$ and the requirement $\underset{n\to \infty }{lim}{\theta }_n=0$. The structures of our proposed step sizes are similar symmetry, and they have positive lower bounds and grow with the iterative count, leading to accelerated convergence;

-

We propose inertial gradient-CQ algorithms with double non-Lipschitz step sizes for solving the $SFP$, and we establish their weak and strong convergence without requiring Lipschitz continuity of $\nabla V_n$ or firm nonexpansiveness of $I-P_{Q_n}$, respectively;

-

We apply our methods to the LASSO problem to demonstrate and validate the theoretical results.

The paper is constructed as follows. In Section 2, some preliminaries are listed. Our proposed algorithms and their convergence results are presented in Section 3 and Section 4. A comparison of the results is described in Section 5.

2. Preliminaries

The symbol ⇀ stands for weak convergence and → represents strong convergence. Let $H_1$ and $H_2$ be real Hilbert spaces. For any sequence $\left\{{\vartheta }_n\right\}\subset H_1$, ${\omega }_{\omega }\left({\vartheta }_n\right)$ denotes the weak $w-$ limit set of $\left\{{\vartheta }_n\right\}$; namely, ${\omega }_{\omega }\left({\vartheta }_n\right)=\left\{{\zeta }_1|\exists \left({\vartheta }_{n_i}\right)\subset \left({\vartheta }_n\right)\mathrm{such}\mathrm{that}{\vartheta }_{n_i}\rightharpoonup {\zeta }_1\right\}$.

Definition 1

([32]). Let $H_1$ be a real Hilbert space and let $V:H_1\to (-\infty ,+\infty )$ be a convex function. An element $\nu \in H_1$ is called the subgradient of V at $\tilde{v}\in H_1$ if

$$⟨\nu ,{\zeta }_1-\tilde{v}⟩\le V\left({\zeta }_1\right)-V\left(\tilde{v}\right),\forall {\zeta }_1\in H_1.$$

The collection of all subgradients of V at $\tilde{v}$ are subdifferentials of the function V at this point, which is denoted by $\partial V\left(\tilde{v}\right)$, i.e.,

$$\partial V\left(\tilde{v}\right)=\{\nu \in H_1:⟨\nu ,{\zeta }_1-\tilde{v}⟩\le V\left({\zeta }_1\right)-V\left(\tilde{v}\right),\forall {\zeta }_1\in H_1\}.$$

Lemma 1.

Let $H_1$ be a real Hilbert space. For all ${\zeta }_1,{\zeta }_2,{\zeta }_3\in H_1$, then

(i) $\parallel \alpha {\zeta }_1+(1-\alpha ){\zeta }_2{\parallel }^2=\alpha \parallel {\zeta }_1{\parallel }^2+(1-\alpha )\parallel {\zeta }_2{\parallel }^2-\alpha (1-\alpha ){\parallel {\zeta }_1-{\zeta }_2\parallel }^2,\forall \alpha \in (0,1).$

(ii) $\parallel {\zeta }_1+{\zeta }_2{\parallel }^2\le {\parallel {\zeta }_1\parallel }^2+2⟨{\zeta }_2,{\zeta }_1+{\zeta }_2⟩.$

(iii) $⟨{\zeta }_1-{\zeta }_2,{\zeta }_1-{\zeta }_3⟩={\displaystyle \frac{1}{2}}\parallel {\zeta }_1-{\zeta }_2{\parallel }^2+{\displaystyle \frac{1}{2}}\parallel {\zeta }_1-{\zeta }_3{\parallel }^2-{\displaystyle \frac{1}{2}}{\parallel {\zeta }_2-{\zeta }_3\parallel }^2.$

Let C be a nonempty closed and convex subset of $H_1$, then the orthogonal projection of ${\zeta }_1$ onto C is defined by

$$P_C{\zeta }_1=\mathrm{argmin}\{\parallel {\zeta }_1-{\zeta }_2\parallel |{\zeta }_2\in C\},\forall {\zeta }_1\in H_1.$$

By the following Lemma 2 (iii), this is a firmly nonexpansive mapping.

Lemma 2

([33]). Let C be a nonempty closed convex subset of a real Hilbert space $H_1$ and let $P_C$ be the metric projection from $H_1$ onto C. Then the following statements hold:

(i) $⟨{\zeta }_1-P_C{\zeta }_1,{\zeta }_2-P_C{\zeta }_1⟩\le 0forall{\zeta }_1\in H_{1}and{\zeta }_2\in C;$

(ii) $\parallel P_C{\zeta }_1-P_C{\zeta }_2\parallel \le \parallel {\zeta }_1-{\zeta }_2\parallel forall{\zeta }_1,{\zeta }_2\in H_1;$

(iii) $\parallel P_C{\zeta }_1-P_C{\zeta }_2{\parallel }^2\le ⟨{\zeta }_1-{\zeta }_2,P_C{\zeta }_1-P_C{\zeta }_2⟩forall{\zeta }_1,{\zeta }_2\in H_1;$

(iv) $\parallel P_C{\zeta }_1-{\zeta }_2{\parallel }^2\le \parallel {\zeta }_1-{\zeta }_2{\parallel }^2-{\parallel {\zeta }_1-P_C{\zeta }_1\parallel }^{2}forall{\zeta }_1\in H_{1}and{\zeta }_2\in C.$

A function $V:H_1\to \mathbb{R}$ is called weakly lower semi-continuous (w-lsc) at $\tilde{v}$ if ${\vartheta }_n\rightharpoonup \tilde{v}$ yields

$$V\left(\tilde{v}\right)\le \underset{n\to \infty }{lim\; inf}V\left({\vartheta }_n\right).$$

Lemma 3

([34]). Let $\left\{{\phi }_n\right\}$ be a sequence of nonnegative numbers fulfilling the following:

$${\phi }_{n+1}\le d_n{\phi }_n+{\psi }_n,\forall n\in \mathbb{N},$$

where $\left\{d_n\right\}$ and $\left\{{\psi }_n\right\}$ are sequences of nonnegative numbers, such that $\left\{d_n\right\}\subset [1,+\infty ),$ ${\displaystyle \sum _{n=1}^{\infty }}(d_n-1)<\infty$ and ${\displaystyle \sum _{n=1}^{\infty }}{\psi }_n<\infty$. Then, ${\displaystyle \underset{n\to \infty }{lim}}{\phi }_n$ exists.

Lemma 4

([9]). Let $H_1$ be a real Hilbert space and let $\left\{{\vartheta }_n\right\}$ be a sequence in $H_1$, such that there exists a nonempty closed and convex subset C of $H_1$ fulfilling the following conditions:

(i) for all $z\in C$, ${\displaystyle \underset{n\to \infty }{lim}}\parallel {\vartheta }_n-z\parallel$ exists;

(ii) any weak cluster point of $\left\{{\vartheta }_n\right\}$ belongs to C.

• Then, there exists $S^{*}\in C$, such that $\left\{{\vartheta }_n\right\}$ converges weakly to $S^{*}.$

Lemma 5

((Lemma 2 [9])). Let $\left\{{\mathsf{\Xi }}_n\right\}\subset [0,\infty )$ and $\left\{b_n\right\}\subset [0,\infty )$ be sequences and such that there $\exists N>0,\forall n\ge N,$

(i) ${\mathsf{\Xi }}_{n+1}-{\mathsf{\Xi }}_n\le {\Re }_n({\mathsf{\Xi }}_n-{\mathsf{\Xi }}_{n-1})+b_n;$

(ii) ${\displaystyle \sum _{n=N}^{\infty }}{b}_n<\infty ;$

(iii) $\left\{{\Re }_n\right\}\subset [0,\beta ],$ where $\beta \in [0,1).$

• Then, $\left\{{\mathsf{\Xi }}_n\right\}$ is a converging sequence and ${\displaystyle \sum _{n=N}^{\infty }}{[{\mathsf{\Xi }}_n-{\mathsf{\Xi }}_{n-1}]}_{+}<\infty ,$ where ${\left[t\right]}_{+}=max\{t,0\}$ (for any $t\in \mathbb{R}$).

Lemma 6

([35]). Let C and Q be closed and convex subsets of real Hilbert spaces $H_1$ and $H_2$, respectively, and $A:H_1\to H_2$ be a bounded linear operator. Let $V:H_1\to \mathbb{R}$ be a function defined by

$$V\left({\zeta }_1\right)={\displaystyle \frac{1}{2}}{\parallel A{\zeta }_1-P_{Q}A{\zeta }_1\parallel }^2,\forall {\zeta }_1\in H_1.$$

The following properties hold:

(i) V is convex and differentiable;

(ii) V is $w-$ lsc on $H_1$;

(iii) $\nabla V\left({\zeta }_1\right)=A^{*}(I-P_Q)A{\zeta }_1$, $\forall {\zeta }_1\in H_1$;

(iv) $\nabla V$ is ${\parallel A\parallel }^2-$ Lipschitz continuous.

Lemma 7

([36]). Suppose that $\left\{{\Pi }_n\right\}$ is a sequence of nonnegative real numbers, such that there $\exists N>0$, $\forall n\ge N,$

$$\begin{array}{c}\begin{array}{c}{\Pi }_{n+1}\le (1-{\pi }_n){\Pi }_n+{\pi }_n{b}_n\hfill \\ {\Pi }_{n+1}\le {\Pi }_n-{\delta }_n+{\mathsf{\Xi }}_n,\hfill \end{array}\hfill \end{array}$$

where $\left\{{\pi }_n\right\}$ is a sequence in $(0,1)$, $\left\{{\delta }_n\right\}$ is a sequence of nonnegative real numbers, and $\left\{b_n\right\}$ and $\left\{{\mathsf{\Xi }}_n\right\}$ are two sequences in $\mathbb{R}$, such that

(i) ${\displaystyle \sum _{n=1}^{\infty }}{\pi }_n=\infty ;$

(ii) ${\displaystyle \underset{n\to \infty }{lim}}{\mathsf{\Xi }}_n=0$;

(iii) ${\displaystyle \underset{i\to \infty }{lim}}{\delta }_{n_i}=0$ yields ${\displaystyle \underset{i\to \infty }{lim\; sup}}{b}_{n_i}\le 0$ for any subsequence $\left\{n_i\right\}$ of $\left\{n\right\}.$

Then, ${\displaystyle \underset{n\to \infty }{lim}}{\Pi }_n=0$.

Lemma 8

([37]). Consider the $SFP$ with the function V as in Lemma 6, and let $\tau >0$ and $S^{*}\in H$. Then, the equivalence of the following statements holds:

(i) The point $S^{*}$ solves the $SFP$;

(ii) The point $S^{*}$ solves the fixed-point equation:

$$S^{*}=P_C(S^{*}-\tau \nabla V\left(S^{*}\right))=P_C(S^{*}-\tau A^{*}(I-P_Q)A{S}^{*});$$

(iii) The point $S^{*}$ solves the variational inequality problem with respect to the gradient of V, namely, find a point ${\zeta }_1\in C$, such that

$$⟨\nabla V\left({\zeta }_1\right),\varsigma -{\zeta }_1⟩\ge 0,\forall \varsigma \in C.$$

3. Weak Convergence

In this section, we introduce an accelerated gradient-CQ algorithm for solving SFP in real Hilbert spaces. Also, its weak convergence is analyzed under conditions that are weaker than those that are typically assumed. Before presenting our algorithm, we list several preliminary conditions below.

(A1) The solution set of $SFP$ is nonempty, i.e., $\Gamma \ne \varnothing .$

(A2) Sets C and Q are defined in (2), and sets $C_n$ and $Q_n$ are defined in (3), the function $V_n$ with its gradient $\nabla V_n$ are defined in Lemma 6.

Now, our proposed algorithm is as follows.

Remark 1.

As shown in Algorithm 1, if ${\vartheta }_{n+1}=z_n=R_n$, then it can be seen from (7) that $R_n=P_{C_n}(R_n-{\tau }_n\nabla V_n\left(R_n\right))$, that is, $R_n\in C_n$. Also, its weak convergence is analyzed under conditions that are weaker than usual. Before describing our algorithm, several conditions are given below. From Lemma 8 (i–ii), we also show $A{R}_n\in Q_n$. Combining sets $C_n$ and $Q_n$, we further see that $c\left(R_n\right)\le 0$ and $q\left(A{R}_n\right)\le 0.$ Thus, it can be concluded from (2) that $R_n\in C$ and $A{R}_n\in Q$. Also, its weak convergence is analyzed under conditions that are weaker than usual. Before presenting our algorithm, several conditions are given below. Thereby, $R_n$ is a solution of the $SFP$.

Algorithm 1: A weakly convergent algorithm for SFP.

Step 0. Take $x_0,x_1\in H_1$, ${\tau }_1>0$, ${\lambda }_1>0$, $\delta \in \left(0,\rho \right)\subset \left(0,1\right)$ and

$\left\{{\Re }_n\right\}\subset [0,\beta )\subset [0,1)$. The sequence $\left\{d_n\right\}$ satisfies

$\left\{d_n\right\}\subset [1,+\infty ),$ ${\displaystyle \sum _{n=1}^{\infty }}(d_n-1)<\infty$.

Step n. Compute

$$\begin{array}{c}{R}_n={\vartheta }_n+{\Re }_n({\vartheta }_n-{\vartheta }_{n-1}),z_n=R_n-{\tau }_n\nabla V_n\left(R_n\right),\hfill \\ {\vartheta }_{n+1}=P_{C_n}(z_n-{\lambda }_n\nabla V_n\left(z_n\right)),\hfill \end{array}$$ (7)

where the stepsizes ${\tau }_n$ and ${\lambda }_n$ are updated via

$${\tau }_{n+1}=\left\{\begin{array}{cc}min\left\{{\displaystyle \frac{\delta V_n\left(R_n\right)}{\parallel \nabla V_n\left(R_n\right){\parallel }^2}},d_n{\tau }_n\right\},\hfill & \mathrm{if}\nabla V_n\left(R_n\right)\ne 0,\hfill \\ d_n{\tau }_n,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$ (8)

and

$${\lambda }_{n+1}=\left\{\begin{array}{cc}min\left\{{\displaystyle \frac{\delta V_n\left(z_n\right)}{\parallel \nabla V_n\left(z_n\right){\parallel }^2}},d_n{\lambda }_n\right\},\hfill & \mathrm{if}\nabla V_n\left(z_n\right)\ne 0,\hfill \\ d_n{\lambda }_n.\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

Stop criterion: If ${\vartheta }_{n+1}=z_n=R_n$, then stop, $R_n$ is the solution to the $SFP$. Also, its weak convergence is analyzed under conditions that are weaker than usual. Before describing our algorithm, several conditions are given below. Otherwise, return to step n.

Remark 2

([38]). In Lemma 9, the condition is easy to compute. Specifically, given two consecutive iteratives ${\vartheta }_n$ and ${\vartheta }_{n-1}$, once can compute ${\vartheta }_{n+1}$ using (7), where the inertial parameter ${\Re }_n$ is chosen to satisfy $0\le {\Re }_n\le {\overline{\Re }}_n$, with

$${\overline{\Re }}_n=\left\{\begin{array}{cc}min\left\{{\displaystyle \frac{{\varrho }_n}{\parallel {\vartheta }_n-{\vartheta }_{n-1}{\parallel }^2}},\beta \right\}\hfill & if{\vartheta }_n\ne {\vartheta }_{n-1},\hfill \\ \beta ,\hfill & otherwise,\hfill \end{array}\right.$$

(9)

where ${\varrho }_n\subset [0,\infty )$ fulfills ${\sum }_{n=1}^{\infty }{\varrho }_n<\infty .$

Lemma 9.

Let the sequence $\left\{{\vartheta }_n\right\}$ be generated by Algorithm 1, and suppose there $\exists N>0$, $\forall n\ge N,$ ${\sum }_{n=N}^{\infty }{\Re }_n{∥{\vartheta }_n-{\vartheta }_{n-1}∥}^2<\infty .$ Then, $\left\{{\vartheta }_n\right\}$ is a bounded sequence.

Now, we illustrate that our proposed step sizes are well-defined when assuming non-Lipschitz continuity of $\nabla V_n$.

Lemma 10.

Let the sequences $\left\{{\tau }_n\right\}$ and $\left\{{\lambda }_n\right\}$ be generated by Algorithm 1. Then, we obtain ${\displaystyle \underset{n\to \infty }{lim}}{\tau }_n=\tau$, ${\displaystyle \underset{n\to \infty }{lim}}{\lambda }_n=\lambda$, $\tau \ge min\left\{{\displaystyle \frac{\delta }{2\parallel A^{*}A\parallel }},{\tau }_1\right\}$ and $\lambda \ge min\left\{{\displaystyle \frac{\delta }{2\parallel A^{*}A\parallel }},{\lambda }_1\right\}$, where ${\tau }_1$ and ${\lambda }_1$ are initial step sizes.

Proof. 

In situation $\nabla V_n\left(R_n\right)\ne 0$, we obtain

$$\begin{array}{c}\begin{array}{c}\parallel \nabla V_n\left(R_n\right){\parallel }^2=⟨\nabla V_n\left(R_n\right),\nabla V_n\left(R_n\right)⟩\hfill \\ =⟨A^{*}(I-P_{Q_n})A{R}_n,A^{*}(I-P_{Q_n})A{R}_n⟩\hfill \\ =⟨(I-P_{Q_n})A{R}_n,A{A}^{*}(I-P_{Q_n})A{R}_n⟩\hfill \\ \le \parallel A^{*}A\parallel ⟨(I-P_{Q_n})A{R}_n,(I-P_{Q_n})A{R}_n⟩\hfill \\ \le \parallel A^{*}A\parallel \parallel (I-P_{Q_n})A{R}_n{\parallel }^2.\hfill \\ =2\parallel A^{*}A\parallel V_n\left(R_n\right),\hfill \end{array}\hfill \end{array}$$

which yields that

$${\displaystyle \frac{\delta V_n\left(R_n\right)}{\parallel \nabla V_n\left(R_n\right){\parallel }^2}}\ge {\displaystyle \frac{\delta }{2\parallel A^{*}A\parallel }}.$$

This further shows that

$$\begin{array}{c}{\tau }_{n+1}=min\left\{{\displaystyle \frac{\delta V_n\left(R_n\right)}{\parallel \nabla V_n\left(R_n\right){\parallel }^2}},d_n{\tau }_n\right\}\ge min\left\{{\displaystyle \frac{\delta }{2\parallel A^{*}A\parallel }},{\tau }_n\right\}\hfill \end{array}$$

where $d_n\ge 1$. Through induction, we see that the sequence $\left\{{\tau }_n\right\}$ has a lower bound $\min \left\{{\displaystyle \frac{\delta }{2\parallel A^{*}A\parallel }},{\tau }_1\right\}$. Using (8), we find that

$${\tau }_{n+1}\le d_n{\tau }_n.$$

Using Lemma 3, it can be seen that ${\displaystyle \underset{n\to \infty }{lim}}{\tau }_n$ exists and we can denote ${\displaystyle \underset{n\to \infty }{lim}}{\tau }_n=\tau$, and as its lower bound is positive, then $\tau >0$. Similarly, we obtain ${\displaystyle \underset{n\to \infty }{lim}}{\lambda }_n=\lambda$ and $\lambda \ge min\left\{{\displaystyle \frac{\delta }{2\parallel A^{*}A\parallel }},{\lambda }_1\right\}>0$. □

We analyze the following theorem without requiring firm-nonexpansiveness of $I-P_{Q_n}$.

Theorem 1.

Assume that Conditions (A1)–(A2) hold and $\exists N>0$, $\forall n\ge N,$ ${\sum }_{n=N}^{\infty }{\Re }_n{∥x_n-x_{n-1}∥}^2<\infty .$ Then, the sequence $\left\{{\vartheta }_n\right\}$ developed by Algorithm 1 converges weakly to a point in the solution set Γ of $SFP$.

Proof. 

Let $\varsigma \in \Gamma$. Since $C\subset C_n$ and $Q\subset Q_n$, we can see that $\varsigma =P_C\left(\varsigma \right)=P_{C_n}\left(\varsigma \right)$ and $A\varsigma =P_Q\left(A\varsigma \right)=P_{Q_n}\left(A\varsigma \right)$. Using Lemma 1 (iii) and Lemma 2 (ii), we show that

$$\begin{array}{c}⟨R_n-\varsigma ,-\nabla V_n\left(R_n\right)⟩=⟨R_n-\varsigma ,A^{*}(P_{Q_n}-I)A{R}_n⟩\hfill \\ =⟨A{R}_n-A\varsigma ,(P_{Q_n}-I)A{R}_n⟩\hfill \\ =⟨A{R}_n-P_{Q_n}A{R}_n+P_{Q_n}A{R}_n-A\varsigma ,(P_{Q_n}-I)A{R}_n⟩\hfill \\ =⟨P_{Q_n}A{R}_n-A\varsigma ,P_{Q_n}A{R}_n-A{R}_n⟩-{\parallel P_{Q_n}A{R}_n-A{R}_n\parallel }^2\hfill \\ ={\displaystyle \frac{1}{2}}\left\{\parallel P_{Q_n}A{R}_n{-A\varsigma \parallel }^2+\parallel P_{Q_n}A{R}_n-A{R}_n{\parallel }^2-{\parallel A{R}_n-A\varsigma \parallel }^2\right\}\hfill \\ - \parallel P_{Q_n}A{R}_n-A{R}_n{\parallel }^2\hfill \\ \le -{\displaystyle \frac{1}{2}}{\parallel P_{Q_n}A{R}_n-A{R}_n\parallel }^2\hfill \\ =-V_n\left(R_n\right),\hfill \end{array}$$

(10)

Similarly, we arrive at

$$⟨z_n-\varsigma ,-\nabla V_n\left(z_n\right)⟩\le -V_n\left(z_n\right).$$

(11)

From (7), (8), and (10), we have

$$\begin{array}{c}\parallel z_n{-\varsigma \parallel }^2={\parallel R_n-{\tau }_n\nabla V_n\left(R_n\right)-\varsigma \parallel }^2\hfill \\ = \parallel R_n{-\varsigma \parallel }^2+{\tau }_n^2{\parallel \nabla V_n\left(R_n\right)\parallel }^2-2{\tau }_n⟨\nabla V_n\left(R_n\right),R_n-\varsigma ⟩\hfill \\ \le \parallel R_n{-\varsigma \parallel }^2+{\tau }_n^2{\parallel \nabla V_n\left(R_n\right)\parallel }^2-2{\tau }_n{V}_n\left(R_n\right)\hfill \\ \le \parallel R_n{-\varsigma \parallel }^2+2{\tau }_n\left({\displaystyle \frac{\delta {\tau }_n}{{\tau }_{n+1}}}-1\right)V_n\left(R_n\right).\hfill \end{array}$$

(12)

Combining (7), (8), (11), (12), and Lemma 2 (iv), we obtain

$$\begin{array}{c}\parallel {\vartheta }_{n+1}{-\varsigma \parallel }^2={\parallel P_{C_n}(z_n-{\lambda }_n\nabla V_n\left(z_n\right))-\varsigma \parallel }^2\hfill \\ \le \parallel z_n-{\lambda }_n\nabla V_n\left(z_n\right){-\varsigma \parallel }^2-{\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2\hfill \\ = \parallel z_n{-\varsigma \parallel }^2+{\lambda }_n^2{\parallel \nabla V_n\left(z_n\right)\parallel }^2-2{\lambda }_n⟨\nabla V_n\left(z_n\right),z_n-\varsigma ⟩\hfill \\ - \parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right){\parallel }^2.\hfill \\ \le \parallel z_n{-\varsigma \parallel }^2+{\lambda }_n^2\parallel \nabla V_n\left(z_n\right){\parallel }^2-2{\lambda }_n{V}_n\left(z_n\right)-{\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2\hfill \\ \le \parallel z_n{-\varsigma \parallel }^2+2{\lambda }_n\left({\displaystyle \frac{\delta {\lambda }_n}{{\lambda }_{n+1}}}-1\right)V_n\left(z_n\right)-{\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2.\hfill \\ \le \parallel R_n{-z\parallel }^2+2{\tau }_n\left({\displaystyle \frac{\delta {\tau }_n}{{\tau }_{n+1}}}-1\right)V_n\left(R_n\right)\hfill \\ + 2{\lambda }_n\left({\displaystyle \frac{\delta {\lambda }_n}{{\lambda }_{n+1}}}-1\right)V_n\left(z_n\right)-{\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2.\hfill \end{array}$$

(13)

From the definition of $R_n$, it follows that

$$\begin{array}{c}\parallel R_n{-\varsigma \parallel }^2={\parallel {\vartheta }_n+{\Re }_n({\vartheta }_n-{\vartheta }_{n-1})-\varsigma \parallel }^2\hfill \\ = \parallel {\vartheta }_n{-\varsigma \parallel }^2+2{\Re }_n⟨{\vartheta }_n-\varsigma ,{\vartheta }_n-{\vartheta }_{n-1}⟩+{\Re }_n^2{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }^2.\hfill \end{array}$$

(14)

Thanks to Lemma 1 (iii), one has

$$⟨{\vartheta }_n-\varsigma ,{\vartheta }_n-{\vartheta }_{n-1}⟩={\displaystyle \frac{1}{2}}\parallel {\vartheta }_n{-\varsigma \parallel }^2+{\displaystyle \frac{1}{2}}\parallel {\vartheta }_n-{\vartheta }_{n-1}{\parallel }^2-{\displaystyle \frac{1}{2}}{\parallel {\vartheta }_{n-1}-z\parallel }^2.$$

Substituting this into (14), we arrive at

$$\begin{array}{c}\parallel R_n{-\varsigma \parallel }^2= \parallel {\vartheta }_n{-\varsigma \parallel }^2+{\Re }_n^2\parallel {\vartheta }_n-{\vartheta }_{n-1}{\parallel }^2+{\Re }_n{\parallel {\vartheta }_n-\varsigma \parallel }^2\hfill \\ + {\Re }_n\parallel {\vartheta }_n-{\vartheta }_{n-1}{\parallel }^2-{\Re }_n{\parallel {\vartheta }_{n-1}-\varsigma \parallel }^2\hfill \\ \le \parallel {\vartheta }_n{-\varsigma \parallel }^2+{\Re }_n(\parallel {\vartheta }_n{-\varsigma \parallel }^2-\parallel {\vartheta }_{n-1}{-\varsigma \parallel }^2)+2{\Re }_n{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }^2.\hfill \end{array}$$

(15)

Using (13) and (15), we can verify that

$$\begin{array}{c}\parallel {\vartheta }_{n+1}{-z\parallel }^2\le {\parallel R_n-\varsigma \parallel }^2+2{\tau }_n\left({\displaystyle \frac{\delta {\tau }_n}{{\tau }_{n+1}}}-1\right)V_n\left(R_n\right)\hfill \\ + 2{\lambda }_n\left({\displaystyle \frac{\delta {\lambda }_n}{{\lambda }_{n+1}}}-1\right)V_n\left(z_n\right)-{\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2.\hfill \\ \le \parallel {\vartheta }_n{-\varsigma \parallel }^2+{\Re }_n(\parallel {\vartheta }_n{-\varsigma \parallel }^2-\parallel {\vartheta }_{n-1}{-\varsigma \parallel }^2)+2{\Re }_n{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }^2\hfill \\ + 2{\tau }_n\left({\displaystyle \frac{\delta {\tau }_n}{{\tau }_{n+1}}}-1\right)V_n\left(R_n\right)+2{\lambda }_n\left({\displaystyle \frac{\delta {\lambda }_n}{{\lambda }_{n+1}}}-1\right)V_n\left(z_n\right)\hfill \\ - \parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right){\parallel }^2.\hfill \end{array}$$

(16)

Since

$$\begin{array}{c}{\displaystyle \underset{n\to \infty }{lim}}\left(1-{\displaystyle \frac{\delta {\tau }_n}{{\tau }_{n+1}}}\right)=1-\delta >1-\rho ,\hfill \\ {\displaystyle \underset{n\to \infty }{lim}}\left(1-{\displaystyle \frac{\delta {\lambda }_n}{{\lambda }_{n+1}}}\right)=1-\delta >1-\rho ,\hfill \end{array}$$

where $\delta \in (0,\rho )\subset (0,1)$. Then, $\exists N\ge 0$, $\forall n\ge N$, $1-{\displaystyle \frac{\delta {\tau }_n}{{\tau }_{n+1}}}>1-\rho$ and $1-{\displaystyle \frac{\delta {\lambda }_n}{{\lambda }_{n+1}}}>1-\rho$. So, $\forall n\ge N$,

$$\begin{array}{c}\parallel {\vartheta }_{n+1}{-z\parallel }^2\le \parallel {\vartheta }_n{-\varsigma \parallel }^2+{\Re }_n(\parallel {\vartheta }_n{-\varsigma \parallel }^2-\parallel {\vartheta }_{n-1}{-\varsigma \parallel }^2)+2{\Re }_n{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }^2\hfill \\ + 2\left(\rho -1\right)({\tau }_n{V}_n\left(R_n\right)+{\lambda }_n{V}_n\left(z_n\right))-{\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2.\hfill \\ \le \parallel {\vartheta }_n{-\varsigma \parallel }^2+{\Re }_n(\parallel {\vartheta }_n{-\varsigma \parallel }^2-\parallel {\vartheta }_{n-1}{-\varsigma \parallel }^2)+2{\Re }_n{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }^2\hfill \end{array}$$

(17)

Now, applying Lemma 5 with

$${\mathsf{\Xi }}_n={\parallel {\vartheta }_n-\varsigma \parallel }^2$$

and

$$b_n=2{\Re }_n{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }^2.$$

Since ${\sum }_{n=N}^{\infty }{\Re }_n{∥{\vartheta }_n-{\vartheta }_{n-1}∥}^2<\infty$, and using Lemma 5, we can see that ${\displaystyle \underset{n\to \infty }{lim}}\parallel {\vartheta }_n-\varsigma \parallel$ exists and

$${\sum }_{n=N}^{\infty }{\left[{∥{\vartheta }_n-\varsigma ∥}^2-{∥{\vartheta }_{n-1}-\varsigma ∥}^2\right]}_{+}<\infty ,$$

where ${\left[t\right]}_{+}=max\{t,0\}.$ As a result,

$$\underset{n\to \infty }{lim}{\left[{∥{\vartheta }_n-\varsigma ∥}^2-{∥{\vartheta }_{n-1}-\varsigma ∥}^2\right]}_{+}=0.$$

Further, it follows from (17) that $\forall n\ge N$,

$$\begin{array}{c}2\left(1-\rho \right)({\tau }_n{V}_n\left(R_n\right)+{\lambda }_n{V}_n\left(z_n\right))+{\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2\hfill \\ \le \parallel {\vartheta }_n{-\varsigma \parallel }^2-\parallel {\vartheta }_{n+1}{-\varsigma \parallel }^2+{\Re }_n(\parallel {\vartheta }_n{-\varsigma \parallel }^2-\parallel {\vartheta }_{n-1}{-\varsigma \parallel }^2)+2{\Re }_n{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }^2\hfill \\ \le \parallel {\vartheta }_n{-\varsigma \parallel }^2-\parallel {\vartheta }_{n+1}{-\varsigma \parallel }^2+{\Re }_n{\left[\parallel {\vartheta }_n{-\varsigma \parallel }^2-{\parallel {\vartheta }_{n-1}-\varsigma \parallel }^2\right]}_{+}+2{\Re }_n{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }^2,\hfill \end{array}$$

which implies that

$$\underset{n\to \infty }{lim}({\tau }_n{V}_n\left(R_n\right)+{\lambda }_n{V}_n\left(z_n\right))=0,$$

which, along with ${\displaystyle \underset{n\to \infty }{lim}}{\tau }_n=\tau >0$ and ${\displaystyle \underset{n\to \infty }{lim}}{\lambda }_n=\lambda >0$, deduces that

$$\underset{n\to \infty }{lim}{V}_n\left(R_n\right)=0\mathrm{and}\underset{n\to \infty }{lim}{V}_n\left(z_n\right)=0.$$

(18)

Moreover, from (17), we have

$$\underset{n\to \infty }{lim}{\parallel {\vartheta }_{n+1}-{\varsigma }_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2=0.$$

(19)

Equation (18) shows that

$$\underset{n\to \infty }{lim}\parallel (I-P_{Q_n})A{R}_n\parallel =0.$$

(20)

Additionally, using (7), (18), and (19), we arrive at

$$\begin{array}{c}\parallel z_n-R_n{\parallel }^2={\tau }_n^2{\parallel \nabla V_n\left(R_n\right)\parallel }^2\hfill \\ ={\displaystyle \frac{{\tau }_n^2}{{\tau }_{n+1}}}{\tau }_{n+1}{\parallel \nabla V_n\left(R_n\right)\parallel }^2\hfill \\ \le {\displaystyle \frac{2\delta {\tau }_n^2}{{\tau }_{n+1}}}{V}_n\left(R_n\right)\hfill \\ \to 0,\mathrm{as}n\to \infty ,\hfill \end{array}$$

which means that

$$\begin{array}{c}\parallel z_n-R_n\parallel \to 0,\mathrm{as}n\to \infty .\hfill \end{array}$$

(21)

and

$$\begin{array}{c}\parallel {\vartheta }_{n+1}-z_n{\parallel }^2\le {\left(\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel +\parallel {\lambda }_n\nabla V_n\left(z_n\right)\parallel \right)}^2\hfill \\ \le 2\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right){\parallel }^2+2{\lambda }_n^2{\parallel \nabla V_n\left(z_n\right)\parallel }^2\hfill \\ =2\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right){\parallel }^2+{\displaystyle \frac{2{\lambda }_n^2}{{\lambda }_{n+1}}}{\lambda }_{n+1}{\parallel \nabla V_n\left(z_n\right)\parallel }^2\hfill \\ \le 2\parallel {\vartheta }_{n+1}-z_n+{\lambda }_n\nabla V_n\left(z_n\right){\parallel }^2+{\displaystyle \frac{4\delta {\lambda }_n^2}{{\lambda }_{n+1}}}{V}_n\left(z_n\right)\hfill \\ \to 0,\mathrm{as}n\to \infty ,\hfill \end{array}$$

which yields that

$$\begin{array}{c}\parallel {\vartheta }_{n+1}-z_n\parallel \to 0,\mathrm{as}n\to \infty ,\hfill \end{array}$$

(22)

where the second inequality in (22) comes from the basic inequality ${(c+d)}^2\le 2c^2+2d^2$. In view of (21) and (22), we have

$$\underset{n\to \infty }{lim}\parallel {\vartheta }_{n+1}-R_n\parallel =0.$$

(23)

Using (7) and (9), we see that $\forall n\ge N$,

$$\parallel R_n-{\vartheta }_n{\parallel }^2={\Re }_n^2\parallel {\vartheta }_n-{\vartheta }_{n-1}{\parallel }^2\le \beta {\Re }_n{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }^2\to 0\mathrm{as}n\to \infty .$$

This shows that

$$\underset{n\to \infty }{lim}\parallel R_n-{\vartheta }_n\parallel =0.$$

(24)

Combining (23) and (24), we observe that

$$\underset{n\to \infty }{lim}\parallel {\vartheta }_{n+1}-{\vartheta }_n\parallel =0.$$

Now, we need to verify that ${\omega }_{\omega }\left({\vartheta }_n\right)\subset \Gamma .$ Let $S^{*}\in {\omega }_{\omega }\left({\vartheta }_n\right)$ be an arbitrary element. Since $\left\{{\vartheta }_n\right\}$ is bounded by Lemma 9, there exists a sequence $\left\{{\vartheta }_{n_i}\right\}$ of $\left\{{\vartheta }_n\right\}$, such that $x_{n_i}\rightharpoonup S^{*}.$ It follows from (24) that $R_{n_i}\rightharpoonup S^{*}\in H_1$. Note that $P_{Q_{n_i}}A{R}_{n_i}\in Q_{n_i}$, one has

$$q\left(A{R}_{n_i}\right)\le ⟨{\zeta }_{n_i},A{R}_{n_i}-P_{Q_{n_i}}A{R}_{n_i}⟩,$$

(25)

where ${\zeta }_{n_i}\in \partial q\left(A{R}_{n_i}\right)$. From the boundedness of $\partial q$, we conclude that $\left\{{\zeta }_{n_i}\right\}$ is bounded. From (20) and (25), we obtain

$$q\left(A{R}_{n_i}\right)\le \parallel {\zeta }_{n_i}\parallel \parallel A{R}_{n_i}-P_{Q_{n_i}}A{R}_{n_i}\parallel \to 0,\mathrm{as}i\to \infty .$$

(26)

Using w-lsc of q again, it implies that

$$q\left(A{S}^{*}\right)\le \underset{i\to \infty }{lim\; inf}q\left(A{R}_{n_i}\right)\le 0.$$

This concludes that $A{S}^{*}\in Q.$

Below, we establish that $S^{*}\in C.$ Using the definition of $C_{n_i}$ and $x_{n_i+1}\in C_{n_i}$, we can verify that

$$c\left(R_{n_i}\right)\le ⟨{\epsilon }_{n_i},R_{n_i}-{\epsilon }_{n_i+1}⟩.$$

(27)

where ${\epsilon }_{n_i}\in \partial c\left(R_{n_i}\right).$ From the boundedness of $\partial c$, $\left\{{\epsilon }_{n_i}\right\}$ is bounded. From (23) and (27), we see that

$$c\left(R_{n_i}\right)\le ∥{\epsilon }_{n_i}\parallel \parallel R_{n_i}-{\epsilon }_{n_i+1}∥\to 0,\mathrm{as}i\to \infty .$$

(28)

From the w-lsc of c, $R_{n_i}\rightharpoonup S^{*}$ and (28), it follows that

$$c\left(S^{*}\right)\le \underset{i\to \infty }{lim\; inf}c\left(R_{n_i}\right)\le 0.$$

Hence, $S^{*}\in C.$ So, we know that $S^{*}\in \Gamma .$ Since the choice of $S^{*}\in {\omega }_{\omega }\left({\vartheta }_n\right)$ is arbitrary, we arrive at ${\omega }_{\omega }\left({\vartheta }_n\right)\subset \Gamma .$ Hence, the results are achieved using Lemma 4. □

4. Strong Convergence

This section presents a strongly convergent algorithm that integrates inertial effects, the relaxed gradient-CQ method [22], and the Halpern-type method [25], combined with newly designed step sizes. The following assumptions are given to prove the convergence of the method.

(A3) Let $\left\{{\varrho }_n\right\}$ be a positive sequence in $[0,\tilde{\theta })$, such that ${\varrho }_n=o\left({\pi }_n\right)$, that is, ${\displaystyle \underset{n\to \infty }{lim}}{\displaystyle \frac{{\varrho }_n}{{\pi }_n}}=0$, where $0<\tilde{\theta }<1$ and $\left\{{\pi }_n\right\}\subset (0,1)$ fulfill ${\displaystyle \sum _{n=1}^{\infty }}{\pi }_n=\infty \mathrm{and}{\displaystyle \underset{n\to \infty }{lim}}{\pi }_n=0.$

The proposed approach has the following form.

Remark 3.

In Algorithm 2, the inertial parameter ${\Re }_n$ is chosen as

$${\Re }_n=\left\{\begin{array}{cc}min\left\{{\displaystyle \frac{{\varrho }_n}{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }},\beta \right\}\hfill & if{\vartheta }_n\ne {\vartheta }_{n-1},\hfill \\ \beta ,\hfill & otherwise,\hfill \end{array}\right.$$

(29)

Algorithm 2: A strongly convergent algorithm for SFP.

Step 0. Take $x_0,x_1\in H_1$, ${\tau }_1>0$, ${\lambda }_1>0$, $\delta \in \left(0,\rho \right)\subset \left(0,1\right)$ and

$\left\{{\Re }_n\right\}\subset [0,\beta )\subset [0,1)$, u is a fixed vector. The sequence $\left\{d_n\right\}$ satisfies

$\left\{d_n\right\}\subset [1,+\infty ),$ ${\displaystyle \sum _{n=1}^{\infty }}(d_n-1)<\infty$.

Step n. Compute

$$\begin{array}{c}{R}_n={\vartheta }_n+{\Re }_n({\vartheta }_n-{\vartheta }_{n-1}),z_n=R_n-{\tau }_n\nabla V_n\left(R_n\right),\hfill \\ {\vartheta }_{n+1}={\pi }_{n}u+(1-{\pi }_n)P_{C_n}(z_n-{\lambda }_n\nabla V_n\left(z_n\right)),\hfill \end{array}$$ (30)

where the step sizes ${\tau }_n$ and ${\lambda }_n$ are updated via

$${\tau }_{n+1}=\left\{\begin{array}{cc}min\left\{{\displaystyle \frac{\delta V_n\left(R_n\right)}{\parallel \nabla V_n\left(R_n\right){\parallel }^2}},d_n{\tau }_n\right\},\hfill & \mathrm{if}\nabla V_n\left(R_n\right)\ne 0,\hfill \\ d_n{\tau }_n,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$ (31)

and

$${\lambda }_{n+1}=\left\{\begin{array}{cc}min\left\{{\displaystyle \frac{\delta V_n\left(z_n\right)}{\parallel \nabla V_n\left(z_n\right){\parallel }^2}},d_n{\lambda }_n\right\},\hfill & \mathrm{if}\nabla V_n\left(z_n\right)\ne 0,\hfill \\ d_n{\lambda }_n,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

Stop criterion: If ${\vartheta }_{n+1}=z_n=R_n$ and $z_n=P_{C_n}(z_n-{\lambda }_n\nabla V_n\left(z_n\right))$, then stop; $R_n$ is a solution of the $SFP$. Otherwise, return to Step n.

In the following, the strong convergence proofs of Algorithm 2 are similar to some weakly convergent proof steps from Algorithm 1 and are thus omitted.

Theorem 2.

Assume that Conditions (A1)–(A3) hold, then the sequence $\left\{{\vartheta }_n\right\}$ generated by Algorithm 2 converges strongly to a point $P_{\Gamma }u$ in the solution set Γ of $SFP$.

Proof We set $\varsigma =P_{\Gamma }u$ and $t_n=P_{C_n}(z_n-{\lambda }_n\nabla V_n\left(z_n\right))$. Using the similar proof as in Theorem 1, we have that $\exists N\ge 0$, $\forall n\ge N$,

$$\begin{array}{c}\parallel t_n{-\varsigma \parallel }^2={\parallel P_{C_n}(z_n-{\lambda }_n\nabla V_n\left(z_n\right))-z\parallel }^2\hfill \\ \le \parallel z_n{-\varsigma \parallel }^2+2{\lambda }_n\left(\rho -1\right)V_n\left(z_n\right)-{\parallel t_n-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2.\hfill \end{array}$$

(32)

and

$$\begin{array}{c}\parallel z_n{-\varsigma \parallel }^2={\parallel R_n-{\tau }_n\nabla V_n\left(R_n\right)-\varsigma \parallel }^2\hfill \\ \le \parallel R_n{-\varsigma \parallel }^2+2{\tau }_n\left(\rho -1\right)V_n\left(R_n\right).\hfill \end{array}$$

(33)

From the definition of $R_n$, it follows that

$$\begin{array}{c}\parallel R_n-\varsigma \parallel =\parallel {\vartheta }_n+{\Re }_n({\vartheta }_n-{\vartheta }_{n-1})-z\parallel \hfill \\ \le \parallel {\vartheta }_n-\varsigma \parallel +{\Re }_n\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel \hfill \\ = \parallel {\vartheta }_n-\varsigma \parallel +{\pi }_n·{\displaystyle \frac{{\Re }_n}{{\pi }_n}}\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel .\hfill \end{array}$$

(34)

By (29), we have ${\Re }_n\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel \le {\varrho }_n$ for all $n\ge 1$, which, along with ${\displaystyle \underset{n\to \infty }{lim}}{\displaystyle \frac{{\varrho }_n}{{\pi }_n}}=0$, implies that

$$\underset{n\to \infty }{lim}{\displaystyle \frac{{\Re }_n}{{\pi }_n}}\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel \le \underset{n\to \infty }{lim}{\displaystyle \frac{{\varrho }_n}{{\pi }_n}}=0.$$

(35)

So, there is a constant $M_1>0$, such that

$${\displaystyle \frac{{\Re }_n}{{\pi }_n}}\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel \le M_1.$$

(36)

Combining (32)–(35) and (36), we derive that $\forall n\ge N$,

$$\parallel t_n-\varsigma \parallel \le \parallel z_n-\varsigma \parallel \le \parallel R_n-\varsigma \parallel \le \parallel {\vartheta }_n-\varsigma \parallel +{\pi }_n{M}_1.$$

(37)

Therefore, $\forall n\ge N$,

$$\begin{array}{c}\begin{array}{c}\parallel {\vartheta }_{n+1}-\varsigma \parallel =\parallel {\pi }_{n}u+(1-{\pi }_n)t_n-\varsigma \parallel \hfill \\ \le {\pi }_n\parallel u-\varsigma \parallel +(1-{\pi }_n)\parallel t_n-\varsigma \parallel \hfill \\ \le {\pi }_n\parallel u-\varsigma \parallel +(1-{\pi }_n)(\parallel {\vartheta }_n-\varsigma \parallel +{\pi }_n{M}_1)\hfill \\ \le {\pi }_n(\parallel u-\varsigma \parallel +M_1)+(1-{\pi }_n)\parallel {\vartheta }_n-\varsigma \parallel \hfill \\ \le max\left\{\parallel {\vartheta }_n-\varsigma \parallel ,(\parallel u-\varsigma \parallel +M_1)\right\}\hfill \\ \le \dots \le max\{\parallel {\vartheta }_n-\varsigma \parallel ,(\parallel u-\varsigma \parallel +M_1\left)\right\}.\hfill \end{array}\hfill \end{array}$$

This means that the sequence $\left\{{\vartheta }_n\right\}$ is bounded. Hence, $\left\{R_n\right\}$ is also bounded. For $\forall n\ge N$, one has

$$\begin{array}{c}\parallel {\vartheta }_{n+1}{-\varsigma \parallel }^2={\parallel (1-{\pi }_n)t_n+{\pi }_{n}u-\varsigma \parallel }^2\hfill \\ = \parallel {\pi }_n(u-\varsigma )+(1-{\pi }_n)(t_n-\varsigma ){\parallel }^2\hfill \\ \le (1-{\pi }_n){\parallel t_n-\varsigma \parallel }^2+2{\pi }_n⟨u-\varsigma ,{\vartheta }_{n+1}-\varsigma ⟩\hfill \\ \le (1-{\pi }_n)(\parallel R_n{-\varsigma \parallel }^2+2{\tau }_n(\rho -1)V_n\left(R_n\right)+2{\lambda }_n(\rho -1)V_n\left(z_n\right)\hfill \\ - \parallel t_n-z_n+{\lambda }_n\nabla V_n\left(z_n\right){\parallel }^2)+2{\pi }_n⟨u-\varsigma ,{\vartheta }_{n+1}-\varsigma ⟩\hfill \\ =(1-{\pi }_n){\parallel R_n-\varsigma \parallel }^2+2(\rho -1)(1-{\pi }_n){\tau }_n{V}_n\left(R_n\right)+2(\rho -1)(1-{\pi }_n){\lambda }_n{V}_n\left(z_n\right)\hfill \\ -(1-{\pi }_n){\parallel t_n-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2+2{\pi }_n⟨u-\varsigma ,{\vartheta }_{n+1}-\varsigma ⟩.\hfill \end{array}$$

(38)

Now, we compute the following estimation:

$$\begin{array}{c}\parallel R_n{-\varsigma \parallel }^2={\parallel {\vartheta }_n+{\Re }_n({\vartheta }_n-{\vartheta }_{n-1})-\varsigma \parallel }^2\hfill \\ = \parallel {\vartheta }_n{-\varsigma \parallel }^2+{\Re }_n^2{\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel }^2+2{\Re }_n⟨{\vartheta }_n-\varsigma ,{\vartheta }_n-{\vartheta }_{n-1}⟩\hfill \\ \le \parallel {\vartheta }_n{-\varsigma \parallel }^2+{\Re }_n^2\parallel {\vartheta }_n-{\vartheta }_{n-1}{\parallel }^2+2{\Re }_n\parallel {\vartheta }_n-\varsigma \parallel \parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel .\hfill \end{array}$$

(39)

Let $M_2={\displaystyle \underset{n\ge N}{sup}}\{\beta \parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel ,\parallel {\vartheta }_n-\varsigma \parallel \}$. Substituting (39) into (38), we find that $\forall n\ge N$,

$$\begin{array}{c}\begin{array}{c}\parallel {\vartheta }_{n+1}{-\varsigma \parallel }^2\le (1-{\pi }_n){\parallel R_n-\varsigma \parallel }^2+2(\rho -1)(1-{\pi }_n){\tau }_n{V}_n\left(R_n\right)+2(\rho -1)(1-{\pi }_n){\lambda }_n{V}_n\left(z_n\right)\hfill \\ -(1-{\pi }_n){\parallel t_n-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2+2{\pi }_n⟨u-\varsigma ,{\vartheta }_{n+1}-\varsigma ⟩.\hfill \\ \le (1-{\pi }_n)(\parallel {\vartheta }_n{-\varsigma \parallel }^2+{\Re }_n^2\parallel {\vartheta }_n-{\vartheta }_{n-1}{\parallel }^2+2{\Re }_n\parallel {\vartheta }_n-\varsigma \parallel \parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel )\hfill \\ + 2(\rho -1)(1-{\pi }_n){\tau }_n{V}_n\left(R_n\right)+2(\rho -1)(1-{\pi }_n){\lambda }_n{V}_n\left(z_n\right)\hfill \\ -(1-{\pi }_n){\parallel t_n-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2+2{\pi }_n⟨u-\varsigma ,{\vartheta }_{n+1}-\varsigma ⟩.\hfill \\ =(1-{\pi }_n)(\parallel {\vartheta }_n{-\varsigma \parallel }^2+{\Re }_n\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel (2\parallel {\vartheta }_n-\varsigma \parallel +{\Re }_n\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel \left)\right)\hfill \\ + 2(\rho -1)(1-{\pi }_n){\tau }_n{V}_n\left(R_n\right)+2(\rho -1)(1-{\pi }_n){\lambda }_n{V}_n\left(z_n\right)\hfill \\ -(1-{\pi }_n){\parallel t_n-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2+2{\pi }_n⟨u-\varsigma ,{\vartheta }_{n+1}-\varsigma ⟩.\hfill \\ \le (1-{\pi }_n){\parallel {\vartheta }_n-\varsigma \parallel }^2+2(\rho -1)(1-{\pi }_n){\tau }_n{V}_n\left(R_n\right)+2(\rho -1)(1-{\pi }_n){\lambda }_n{V}_n\left(z_n\right)\hfill \\ -(1-{\pi }_n)\parallel t_n-z_n+{\lambda }_n\nabla V_n\left(z_n\right){\parallel }^2+3M_2{\Re }_n\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel \hfill \\ + 2{\pi }_n⟨u-\varsigma ,{\vartheta }_{n+1}-\varsigma ⟩.\hfill \end{array}\hfill \end{array}$$

After arrangement, $\forall n\ge N$,

$$\begin{array}{c}\parallel {\vartheta }_{n+1}{-\varsigma \parallel }^2\le (1-{\pi }_n){\parallel {\vartheta }_n-\varsigma \parallel }^2+2(\rho -1)(1-{\pi }_n){\tau }_n{V}_n\left(R_n\right)+2(\rho -1)(1-{\pi }_n){\lambda }_n{V}_n\left(z_n\right)\hfill \\ -(1-{\pi }_n)\parallel t_n-z_n+{\lambda }_n\nabla V_n\left(z_n\right){\parallel }^2+3M_2{\Re }_n\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel \hfill \\ + 2{\pi }_n⟨u-\varsigma ,{\vartheta }_{n+1}-\varsigma ⟩.\hfill \end{array}$$

(40)

Using Lemma 7, we see that $\forall n\ge N$, let

$$\begin{array}{c}\begin{array}{c}{\Pi }_n={\parallel {\vartheta }_n-\varsigma \parallel }^2;\hfill \\ {\mathsf{\Xi }}_n=3M_2{\pi }_n{\displaystyle \frac{{\Re }_n}{{\pi }_n}}\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel +2{\pi }_n⟨u-\varsigma ,{\vartheta }_{n+1}-\varsigma ⟩;\hfill \\ b_n=2⟨u-\varsigma ,{\vartheta }_{n+1}-\varsigma ⟩+3M_2{\displaystyle \frac{{\Re }_n}{{\pi }_n}}\parallel {\vartheta }_n-{\vartheta }_{n-1}\parallel ;\hfill \\ {\delta }_n=(1-{\pi }_n)2{\tau }_n(1-\rho )V_n\left(R_n\right)+2(\rho -1)(1-{\pi }_n){\lambda }_n{V}_n\left(z_n\right)+(1-{\pi }_n){\parallel t_n-z_n+{\lambda }_n\nabla V_n\left(z_n\right)\parallel }^2.\hfill \end{array}\hfill \end{array}$$

So, (40) is reduced to the following inequalities

$$\begin{array}{c}\begin{array}{c}{\Pi }_{n+1}\le (1-{\pi }_n){\Pi }_n+{\pi }_n{b}_n,n\ge N,\hfill \\ {\Pi }_{n+1}\le {\Pi }_n-{\delta }_n+{\mathsf{\Xi }}_n,\hfill \end{array}\hfill \end{array}$$

Let $\left\{n_i\right\}\subset \left\{n\right\}$ be a sequence and assume that

$$\underset{i\to \infty }{lim}{\delta }_{n_i}=0.$$

This means

$$\begin{array}{c}\begin{array}{c}{\displaystyle \underset{i\to \infty }{lim}}\left\{(1-{\alpha }_{n_i})(2{\tau }_{n_i}(1-\rho )V_{n_i}\left(R_{n_i}\right)+2(\rho -1){\lambda }_{n_i}{V}_{n_i}\left(z_{n_i}\right)+\parallel t_{n_i}-z_{n_i}+{\lambda }_{n_i}\nabla V_{n_i}\left(z_{n_i}\right){\parallel }^2)\right\}=0.\hfill \end{array}\hfill \end{array}$$

By ${\displaystyle \underset{i\to \infty }{lim}}{\tau }_{n_i}=\tau >0$, ${\displaystyle \underset{i\to \infty }{lim}}{\lambda }_{n_i}=\lambda >0$ and ${\displaystyle \underset{i\to \infty }{lim}}{\alpha }_{n_i}=0$, we deduce

$$\begin{array}{c}{\displaystyle \underset{i\to \infty }{lim}}{V}_{n_i}\left(R_{n_i}\right)=0,{\displaystyle \underset{i\to \infty }{lim}}{V}_{n_i}\left(z_{n_i}\right)=0\hfill \\ \mathrm{and}{\displaystyle \underset{i\to \infty }{lim}}{\parallel t_{n_i}-z_{n_i}+{\lambda }_{n_i}\nabla V_{n_i}\left(z_{n_i}\right)\parallel }^2=0,\hfill \end{array}$$

(41)

which implies that

$$\begin{array}{c}{\displaystyle \underset{i\to \infty }{lim}}\parallel (I-P_{Q_{n_i}})A{R}_{n_i}\parallel =0.\hfill \end{array}$$

(42)

Similarly to Theorem 1, we can prove that ${\omega }_{\omega }\left({\vartheta }_{n_i}\right)\subset \Gamma .$ Hence, there exists a subsequence $\left\{{\vartheta }_{n_{i_j}}\right\}$ of $\left\{{\vartheta }_{n_i}\right\}$, such that ${\vartheta }_{n_{i_j}}\rightharpoonup S^{*}\in \Gamma$.

• From Lemma 2 (i), we obtain

$$\begin{array}{c}{\displaystyle \underset{i\to \infty }{lim\; sup}}⟨u-\varsigma ,{\vartheta }_{n_i}-\varsigma ⟩={\displaystyle \underset{j\to \infty }{lim}}⟨u-\varsigma ,{\vartheta }_{n_{i_j}}-\varsigma ⟩\hfill \\ =⟨u-\varsigma ,S^{*}-\varsigma ⟩\hfill \\ \le 0.\hfill \end{array}$$

(43)

From (A3), we have

$$\parallel {\vartheta }_{n_i}-R_{n_i}\parallel ={\Re }_{n_i}\parallel {\vartheta }_{n_i}-{\vartheta }_{n_i-1}\parallel ={\pi }_{n_i}{\displaystyle \frac{{\Re }_{n_i}}{{\pi }_{n_i}}}\parallel {\vartheta }_{n_i}-{\vartheta }_{n_i-1}\parallel \to 0,asi\to \infty .$$

(44)

Using the same analysis as in (21), we can show that

$$\begin{array}{c}{\displaystyle \underset{i\to \infty }{lim}}\parallel z_{n_i}-R_{n_i}\parallel =0.\hfill \end{array}$$

(45)

From (44) and (45), we derive that

$$\parallel {\vartheta }_{n_i}-z_{n_i}\parallel \le \parallel {\vartheta }_{n_i}-R_{n_i}\parallel + \parallel z_{n_i}-R_{n_i}\parallel \to 0,asi\to \infty .$$

(46)

From the convexity of ${\parallel ·\parallel }^2$ and (41), we arrive at

$$\begin{array}{c}\parallel {\vartheta }_{n_i+1}-z_{n_i}{\parallel }^2={\parallel {\pi }_{n_i}u+(1-{\pi }_{n_i})t_{n_i}-z_{n_i}\parallel }^2\hfill \\ \le {\pi }_{n_i}\parallel u-{\varsigma }_{n_i}{\parallel }^2+(1-{\pi }_{n_i}){\parallel t_{n_i}-z_{n_i}\parallel }^2\hfill \\ \le {\pi }_{n_i}\parallel u-{\varsigma }_{n_i}{\parallel }^2+2(1-{\pi }_{n_i}){\parallel t_{n_i}-z_{n_i}+{\lambda }_{n_i}\nabla V_{n_i}\left(z_{n_i}\right)\parallel }^2\hfill \\ + 2(1-{\pi }_{n_i}){\lambda }_{n_i}^2{\parallel \nabla V_{n_i}\left(z_{n_i}\right)\parallel }^2\hfill \\ \le {\pi }_{n_i}\parallel u-{\varsigma }_{n_i}{\parallel }^2+2(1-{\pi }_{n_i}){\parallel t_{n_i}-z_{n_i}+{\lambda }_{n_i}\nabla V_{n_i}\left(z_{n_i}\right)\parallel }^2\hfill \\ + 4\delta (1-{\pi }_{n_i}){\displaystyle \frac{{\lambda }_{n_i}^2}{{\lambda }_{n_i+1}}}{V}_{n_i}\left(z_{n_i}\right)\hfill \\ \to 0,asi\to \infty .\hfill \end{array}$$

(47)

So, we arrive at

$$\begin{array}{c}\parallel {\vartheta }_{n_i+1}-{\vartheta }_{n_i}\parallel \le \parallel {\vartheta }_{n_i+1}-z_{n_i}\parallel + \parallel {\vartheta }_{n_i}-z_{n_i}\parallel \hfill \\ \to 0,asi\to \infty .\hfill \end{array}$$

(48)

Combining (43) and (48), we obtain

$$\begin{array}{c}\hfill \begin{array}{c}{\displaystyle \underset{i\to \infty }{lim\; sup}}\left(⟨u-\varsigma ,{\vartheta }_{n_i+1}-z⟩+3M_2{\displaystyle \frac{{\Re }_{n_i}}{{\pi }_{n_i}}}\parallel {\vartheta }_{n_i}-{\vartheta }_{n_i-1}\parallel \right)\hfill \\ \le {\displaystyle \underset{i\to \infty }{lim\; sup}}⟨u-\varsigma ,{\vartheta }_{n_i+1}-{\vartheta }_{n_i}⟩+{\displaystyle \underset{i\to \infty }{lim\; sup}}⟨u-\varsigma ,{\vartheta }_{n_i}-\varsigma ⟩\hfill \\ + 3M_2{\displaystyle \underset{i\to \infty }{lim\; sup}}{\displaystyle \frac{{\Re }_{n_i}}{{\pi }_{n_i}}}\parallel {\vartheta }_{n_i}-{\vartheta }_{n_i-1}\parallel \hfill \\ \le 0.\hfill \end{array}\end{array}$$

Using the above results, one has

$$\underset{i\to \infty }{lim\; sup}{b}_{n_i}\le 0.$$

By Lemma 7, it can be found that the sequence $\left\{{\vartheta }_n\right\}$ converges strongly to $z=P_{\Gamma }\left(u\right)$.

5. Numerical Experiments

In the first numerical example, we compare our Algorithm 1 with some weakly convergent algorithms, including Gibali et al.’s Algorithm 3.1 (shortly, GAlg.3.1) [6] and Sahu et al.’s Algorithm 3.1 (SAlg.3.1) [9]. In the second example, we compare Algorithm 2 with Ma and Liu’s Algorithm 3.1 (shortly, MLAlg.3.1) [11] and Ma et al.’s Algorithm 3 (shortly, Alg.3) [39]. These experiments were cinducted in MATLAB R2017a on a PC Desktop Intel(R) Core(TM) i7-6700 CPU @ 3.40 GHZ computer with RAM 8.00 GB.

Example 1.

LASSO problem [9,40,41,42]

Here, the following LASSO problem was changed to model the $SFP$, and it recovered the sparse signal.

$$min\left\{{\displaystyle \frac{1}{2}}{\parallel A\vartheta -b\parallel }^2:\vartheta \in {\mathbb{R}}^N,{\parallel \vartheta \parallel }_1\le \kappa \right\},$$

(49)

where $A\in {\mathbb{R}}^{M\times N},M<N,$$b\in {\mathbb{R}}^M$ and $\kappa >0.$

Now, we need to find a sparse solution for the $SFP$. Here, the system A is produced from a standard normal distribution with a mean zero and unit variance. The true sparse signal $S^{*}$ is created from uniform distribution at an interval of $[-2,2]$ with a random k position that is nonzero, while the remainder is maintained at zero.The sample data are $b=A{S}^{*}$.

Certain assumptions are imposed on matrix A, where the solution to (49) is equal to the ${\ell }_0-$ norm solution of the underdetermined linear system. Further, $SFP$ is changed into closed convex sets $C=\left\{z\right|\parallel z{\parallel }_1\le \kappa \},\kappa =k,$ and $Q=\left\{b\right\}$. Since the projection onto C has no a closed form solution, the subgradient projection is used. We further introduce a convex function $c\left(z\right)={\parallel z\parallel }_1-\kappa$ and

$$C_n=\{z:c\left({\vartheta }_n\right)+⟨{\epsilon }_n,\varsigma -{\vartheta }_n⟩\le 0\},$$

where ${\epsilon }_n\in \partial c\left({\vartheta }_n\right)$. Meanwhile, the orthogonal projection of a point $z\in {\mathbb{R}}^N$ onto $C_n$ is the following

$$P_{C_n}\left(\varsigma \right)=\left\{\begin{array}{c}\varsigma ,ifc\left({\vartheta }_n\right)+⟨{\epsilon }_n,\varsigma -{\vartheta }_n⟩\le 0,\hfill \\ \varsigma -{\displaystyle \frac{c\left({\vartheta }_n\right)+⟨{\epsilon }_n,\varsigma -{\vartheta }_n⟩}{\parallel {\epsilon }_n{\parallel }^2}}{\epsilon }_n,otherwise,\hfill \end{array}\right.$$

The subdifferential $\partial c$ at ${\vartheta }_n$ is

$$\partial c\left({\vartheta }_n\right)=\left\{\begin{array}{cc}1,\hfill & if{\vartheta }_n>0,\hfill \\ [-1,1]\hfill & if{\vartheta }_n=0,\hfill \\ -1,\hfill & if{\vartheta }_n<0.\hfill \end{array}\right.$$

To ensure all algorithms run efficiently, they were initialized at the start through the following

$$E_n={\displaystyle \frac{\parallel {\vartheta }_n-S^{*}\parallel }{max\{1,\parallel {\vartheta }_n\parallel \}}}.$$

We used this to test how the algorithms behaved under different $M,Nandk$, while using the same iterative numbers (3000). Meanwhile, $E_n$ is reported in Table 1.

Table 1. Results for Example 1.

$(\mathit{M},\mathit{N},\mathit{k})$	Alg.1		SAlg.3.1		GAlg.3.1
	$E_n$	time	$E_n$	time	$E_n$	time
(240, 1024, 30)	1.8479 × 10−10	0.9192	2.5269 × 10−9	1.3943	1.3316 × 10−5	1.8579
(480, 2048, 60)	1.1073 × 10−14	4.5840	6.8123 × 10−12	6.7324	3.3606 × 10−6	8.8775
(720, 3072, 90)	1.0120 × 10−14	27.5802	1.2631 × 10−12	40.6743	1.7002 × 10−6	53.7751
(960, 4096, 120)	1.0123 × 10−14	50.4016	4.7432 × 10−13	75.4683	7.8236 × 10−7	100.9057
(1200, 5120, 150)	9.1161 × 10−15	81.3778	3.0586 × 10−14	122.1031	6.1916 × 10−7	162.5137

In the following, the problem is the recovery of the signal $S^{*}$; so, we take $M=240, N=1024, k=30$, $M\times N$ matrix A is randomly obtained with independent samples of a standard Gaussian distribution. In detail, the original signal $S^{*}$ contains 30 randomly placed $\pm 1$ spikes. All of the algorithms start their iterations with $x_0=0$, and the following mean square error (MSE) is defined for measuring the accuracy of the recovery:

$$MSE={\displaystyle \frac{1}{N}}{\parallel {\vartheta }_n-S^{*}\parallel }^2.$$

For SAlg.3.1, we take ${\mathsf{\Xi }}_n=0.1$ and $\beta =0.3$; for GAlg.3.1, we choose ${\mathsf{\Xi }}_n=0.1$, $\beta =0.3$ and ${\varrho }_n={\displaystyle \frac{1}{n^2}}$; and for Alg.1, we set ${\tau }_1={\lambda }_1=0.3, \delta =0.3,$ $d_n={\displaystyle \frac{1}{{(n+1)}^{1.1}}}+1$, ${\psi }_n=0$, ${\varrho }_n={\displaystyle \frac{1}{n^2}}$ and $\beta =0.3$.

Remark 4.

From Table 1, we see that our proposed algorithm (Alg.1) takes less CPU time to obtain a smaller $E_n$ compared with algorithms SAlg.3.1 and GAlg.3.1 in different situations.

Figure 1 shows the recovery results of all algorithms, and includes the original signal, MSE, and iterative time.

Figure 1. Comparison of signal processing.

Remark 5.

From Figure 1, we know that our proposed Algorithm 1 uses less CPU time and smaller MSE to achieve iterative better numbers (3000) than SAlg.3.1 and GAlg.3.1.

Example 2

([43,44]). Let $H_1=H_2=L^2\left([0,1]\right)$ with norm $\parallel {\zeta }_1{\parallel }_{L^2}={\left({\int }_0^1{\zeta }_1{\left(t\right)}^{2}dt\right)}^{{\displaystyle \frac{1}{2}}}$ and inner product $⟨{\zeta }_1,{\zeta }_2⟩={\int }_0^1{\zeta }_1\left(t\right){\zeta }_2\left(t\right)dt,{\zeta }_1,{\zeta }_2\in L^2\left([0,1]\right)$. Let $C=\{{\zeta }_1\in L^2\left([0,1]\right):\parallel {\zeta }_1{\parallel }_{L^2}\le 1\}$ and $Q=\{{\zeta }_1\in L^2\left([0,1]\right):⟨{\zeta }_1,t⟩=0\}$. Set $A{\zeta }_1\left(t\right)={\displaystyle \frac{{\zeta }_1\left(t\right)}{2}}$. We observe that $0\in \Gamma$, and so $\Gamma \ne \varnothing$. For MLAlg. 3.1, we take ${\lambda }_1=0.0003$, $\delta =0.4,$ $\beta =0.3$, $d_n={\displaystyle \frac{1}{{(n+1)}^{1.1}}}+1$, ${\psi }_n=0$, ${\pi }_n={10}^{-4}n$, ${\varrho }_n={\displaystyle \frac{1}{n^2}}$, and $u={\vartheta }_0$. For Alg. 2, we choose ${\lambda }_1=0.6,{\tau }_1=0.6,\delta =0.9$, $d_n={\displaystyle \frac{10}{{(n+1)}^{1.1}}}+1$, ${\psi }_n=0$, $\beta =0.3$, ${\pi }_n={\displaystyle \frac{{10}^{-4}}{n}}$, ${\varrho }_n={\displaystyle \frac{1}{n^2}}$ and $u=x_0$, which we use here. For all of the algorithms, the stopping criterion is denoted by $\parallel {\vartheta }_{n+1}-{\vartheta }_n{\parallel }_{L^2}<ϵ$. Now, the starting points can be one of the following.

Case 1: ${\vartheta }_0=3t^4+t-6,{\vartheta }_1=5t$,

Case 2: ${\vartheta }_0=e^t-8,{\vartheta }_1=5t$.

The obtained results are recorded in Table 2.

Table 2. Results for Example 2.

Cases	$\mathit{ϵ}$	Alg. 2		MLAlg. 3.1
		iter.	time	iter.	time
Case 1	${10}^{-2}$	7	0.1751	7	0.2262
	${10}^{-3}$	9	0.2418	10	0.7618
Case 2	${10}^{-2}$	7	0.1680	7	0.2330
	${10}^{-3}$	9	0.2610	9	0.2839

In this experiment, the involved projections on sets C and Q have the following formulas, that is,

$$P_C\left(\vartheta \right)=\left\{\begin{array}{cc}{\displaystyle \frac{\vartheta }{{\parallel \vartheta \parallel }_{L^2}}},\hfill & {if\parallel \vartheta \parallel }_{L^2}>1,\hfill \\ \vartheta ,\hfill & {if\parallel \vartheta \parallel }_{L^2}\le 1.\hfill \end{array}\right.and{P}_Q\left(\vartheta \right)=\vartheta -{\displaystyle \frac{⟨t,\vartheta ⟩}{{\parallel t\parallel }_{L^2}}}t.$$

(50)

Remark 6.

We see from Table 2 that our proposed Algorithm 2 fulfilled the stopping criterion at a shorter iterative time and with fewer iterations than MLAlg. 3.1.

Example 3.

The following split feasibility problem is proposed in [9,13,45,46]:

$$\begin{array}{c}\hfill \begin{array}{c}Find{z}^{*}\in CsuchthatA{z}^{*}\in Q,\hfill \end{array}\end{array}$$

where $A\in {\mathbb{R}}^{20\times 10}$ is a matrix. The set $C=\left\{y\in {\mathbb{R}}^{10}:c\left(y\right)\le 0\right\},$ where

$$c\left(y\right)=-y_1+y_2^2+\dots +y_{10}^2,$$

and $Q=\{z\in {\mathbb{R}}^{20}:q\left(z\right)\le 0\},$ where

$$q\left(z\right)=z_1+z_2^2+\dots +z_{20}^2-1.$$

Notice that C is the set above the function $y_1=y_2^2+\dots +y_{10}^2$ and Q is the set below the function $z_1=-z_2^2-\dots -z_{20}^2+1.$ Every element of A is randomly selected in $(0,1)$, fulfilling

$$A\left(C\right)\cap Q\ne \varnothing .$$

For all of the algorithms, we adopted the same starting points $x_0\in {\mathbb{R}}^{10}and{x}_1\in {\mathbb{R}}^{10}$, which were generated randomly, every element in these two starting points is in $(0,1).$

For Alg.3, we used ${\tau }_1=0.01,\delta =0.3,$ $d_n={\displaystyle \frac{22}{{(n+1)}^{1.1}}}+1$, ${\pi }_n={\displaystyle \frac{1}{n}}$, ${\varrho }_n={\displaystyle \frac{1}{n^3}}$, $\beta =0.3$ and $\eta =0.5,$ which was suggested in [39].

For Alg.2, we suggest ${\tau }_1=0.01, {\lambda }_0=0.6,\delta =0.49,$ $d_n={\displaystyle \frac{22}{{(n+1)}^{1.1}}}+1$, ${\pi }_n={\displaystyle \frac{1}{n}}$, ${\varrho }_n={\displaystyle \frac{1}{n^3}}$ and $\beta =0.3$.

Remark 7.

From Table 3, we found that Alg.2 had fewer iterations than Alg.3; however, its iterative time was much longer than Alg.3.

Table 3. Results for Example 3.

$(\mathit{n},\mathit{m})$	Alg.3		Alg.2
	iter.	time	iter.	time
(10, 20)	19	7.5570 × 10−4	17	7.4960 × 10−4
(20, 20)	20	6.6100 × 10−4	19	4.3310 × 10−4
(30, 20)	20	0.0058	18	0.0061

6. Conclusions

Our work establishes the convergence of the proposed algorithms under conditions weaker than the usual assumptions of Lipschitz continuity for $\nabla V_n$ and firm nonexpansiveness for $I-P_{Q_n}$, which are typically associated with the SFP. However, the addition of double inertia to Algorithm 1 makes it challenging to establish a new weak convergence result. Therefore, we leave the investigation of how double inertia affects the effectiveness and convergence analysis of the proposed algorithms for future work.