Modified Relaxed CQ Iterative Algorithms for the Split Feasibility Problem †

The split feasibility problem models inverse problems arising from phase retrievals problems and intensity-modulated radiation therapy. For solving the split feasibility problem, Xu proposed a relaxed CQ algorithm that only involves projections onto half-spaces. In this paper, we use the dual variable to propose a new relaxed CQ iterative algorithm that generalizes Xu’s relaxed CQ algorithm in real Hilbert spaces. By using projections onto half-spaces instead of those onto closed convex sets, the proposed algorithm is implementable. Moreover, we present modified relaxed CQ algorithm with viscosity approximation method. Under suitable conditions, global weak and strong convergence of the proposed algorithms are proved. Some numerical experiments are also presented to illustrate the effectiveness of the proposed algorithms. Our results improve and extend the corresponding results of Xu and some others.


Introduction
Throughout this paper, we always assume that H is a real Hilbert space with inner product •, • and norm • .Let I denote the identity operator on H. Let C and Q be nonempty closed convex subset of real Hilbert spaces H 1 and H 2 , respectively.
The split feasibility problem can mathematically be formulated as the problem of finding a point u * ∈ C with the property where A : H 1 → H 2 is a bounded linear operator.The SFP (SFP) in finite-dimensional Hilbert spaces was first introduced by Censor and Elfving [1] for modeling inverse problems which arise from phase retrievals and medical image reconstruction [2], with particular progress in intensity-modulated radiation therapy [3,4].It has been found that the SFP can also be used in the air traffic flow management problems.Many researchers studied the SFP and introduced various algorithms to solve it (see [5][6][7][8][9][10][11][12][13][14][15] and references therein).
The original algorithm introduced in [1] involves the computation of the inverse A −1 (assuming the existence of the inverse of A) and thus does not become popular.A more popular algorithm that solves the SFP (Equation ( 1)) seems to be the following CQ algorithm of Byrne [2,16]: where P C and P Q are the (orthogonal) projections onto C and Q, respectively, and A * is the adjoint of A and µ ∈ (0, 2/λ) with λ being the spectral radius of the operator A * A. The CQ algorithm only involves the computations of the projections P C and P Q onto the sets C and Q, respectively, and is therefore implementable in the case where P C and P Q have closed-form expressions (e.g., C and Q are the closed balls or half-spaces).It remains however a challenge how to implement the CQ algorithm in the case where the projections P C and/or P Q fail to have closed-form expressions though theoretically we can prove (weak) convergence of the algorithm.We assume that the SFP (Equation ( 1)) is consistent, and use Φ to denote the solution set of the SFP (Equation ( 1)), i.e., Φ = {u ∈ C : Au ∈ Q}.
Thus, the set Φ is closed, convex and nonempty.
The CQ algorithm is found to be a gradient-projection method (GPM) in convex minimization (it is also a special case of the proximal forward-backward splitting method).We can reformulate the SFP (Equation ( 1)) as an optimization problem [17].We may introduce the (convex) objective function and consider the convex minimization problem The objective function g is continuously differentiable with gradient given by Because I − P Q is (firmly) nonexpansive, we obtain that ∇g is Lipschitz continuous with Lipschitz constant L = A 2 .It is well known that the gradient-projection algorithm (GPM), for solving the minimization problem in Equation ( 4), generates the following iterative sequence {u n }: where µ is chosen in the interval (0, 2/L) with L being the Lipschitz constant of ∇g.For solving the problem in Equation ( 4), the GPM with gradient ∇g given as in Equation ( 5) is the CQ algorithm in Equation (2).By Equation (4), the SFP (Equation ( 1)) can be written as the following convex separable minimization problem: min where g(u) is defined by Equation ( 3) and ι C (u) is an indicator function of the set C defined by Chen et al. [18] designed and discussed an efficient algorithm for minimizing the sum of two proper lower semi-continuous convex functions, i.e., min u∈R n g 1 (u) + g 2 (u), (9) where g 1 , g 2 ∈ Γ 0 (R n ) (all proper lower semi-continuous convex functions from R n to (−∞, +∞]) and g 2 is differentiable on R n with 1/β-Lipschitz continuous gradient for some β ∈ (0, +∞).For g ∈ Γ 0 (R n ) and ρ ∈ (0, +∞), the proximal operator of g with order ρ, denoted by prox ρg , is defined by: for each To solve the convex separable problem in Equation ( 9), they obtained the following fixed point formulation: the point u * is a solution of Equation ( 9) if and only if there exists e * ∈ R n such that where λ > 0 and µ > 0. They introduced the following Picard iterative sequence: It was shown [18] that, under appropriate conditions, the sequence {u n } converges to a solution of the problem in Equation (9).Moreover, u is the primal variable and e is the dual variable of the primal-dual form (see [18]) related to Equation (9).
For solving the SFP (Equation ( 1)), we note that the CQ algorithm and many related iterative algorithms (see [19][20][21][22][23][24]) only involves the computations of the projections P C and P Q onto the sets C and Q, respectively, and is therefore implementable in the case where P C and P Q have closed-form expressions.However, in some cases it is impossible or needs too much work to exactly compute an orthogonal projection.Therefore, if it is the case, the efficiency of projection type methods will be seriously affected.To overcome this difficulty, Fukushima [25] suggested a so-called relaxed projection method to calculate the projection onto a level set of a convex function by computing a sequence of projections onto half-spaces containing the original level set.Theoretical analysis and numerical experiments show the efficiency of his method.
Let C and Q be level sets of convex functions, i.e., where c : H 1 → R and q : H 2 → R are convex and lower semi-continuous functions with the subdifferentials where ξ n ∈ ∂c(u n ), and where η n ∈ ∂q(Au n ).Obviously, C n and Q n are half-spaces and the projections onto half-spaces C n and Q n have closed forms.
In the setting of finite-dimensional spaces, relaxed projection method was followed by Yang [26], who introduced the following relaxed CQ algorithms for solving the SFP (Equation ( 1)) where the closed convex subsets C and Q are level sets of convex functions: where µ ∈ (0, 2/L) with L being the largest eigenvalue of matrix A T A, C n and Q n are given in Equations ( 13) and ( 14), respectively.Due to the special form of C n and Q n , the proposed algorithm can be easily implemented.
Recently, for the purpose of generality, the SFP (Equation ( 1)) is studied in a more general setting.For instance, Xu [27] considered the SFP (Equation ( 1)) where H 1 and H 2 are infinite-dimensional Hilbert spaces.Xu [27] proposed the following relaxed CQ algorithm where C and Q are given in Equation ( 12): where µ ∈ (0, 2/ A 2 ), C n and Q n are given in Equations ( 13) and ( 14), respectively.Since the projections P C n and P Q n have closed-form expressions, the above relaxed CQ algorithm is implementable.In [27], the relaxed CQ algorithm has the weak convergence result.He and Zhao [28] introduced a Halpern-type relaxed CQ algorithm such that the strong convergence is guaranteed.Some relaxed algorithms have been proposed to solve the SFP (Equation ( 1)) (see [29][30][31]).Inspired and motivated by the works mentioned above, for solving the SFP (Equation ( 1)) in real Hilbert spaces, we use the dual variable to propose a new relaxed CQ iterative algorithm: where e 0 and u 0 ∈ H 1 are arbitrarily chosen, 0 < λ ≤ 1 and 0 < µ n ≤ 2 A 2 .Taking λ = 1, the proposed algorithm in Equation ( 17) becomes the relaxed CQ algorithm in Equation ( 16) (Xu [27]).Moreover, we present modified relaxed CQ algorithm with viscosity approximation method.Proposed two relaxed CQ iterative algorithms which only involve orthogonal projections onto half-spaces, so that the algorithms are implementable.Under suitable conditions, global weak and strong convergence of the proposed algorithms are proved.Some numerical experiments are also presented to illustrate the effectiveness of the proposed algorithms.Our results improve and extend the corresponding results of Xu and some others.
The rest of this paper is organized as follows.In the next section, some necessary concepts and important facts are collected.The weak convergence theorem of the proposed algorithm is established in Section 3. In Section 4, we modify the proposed algorithm by viscosity method so that it has strong convergence result.Finally, we give some numerical experiments to illustrate the efficiency of the proposed iterative methods.

Preliminaries
In this paper, we use → and to denote the strong convergence and weak convergence, respectively.We use ω w (u n ) = {u : ∃u n j u} to stand for the weak ω-limit set of {u n }.
It is obvious that, if h is a ρ-contraction, then I − h is a (1-ρ)-strongly monotone mapping.Recall the variational inequality problem [32] is to find a point u * ∈ C such that Fu * , u − u * ≥ 0 for all u ∈ C, where C is a nonempty closed convex subset of H and F : C → H is a nonlinear operator.It is well known that [33] if F : C → H is a Lipschitzian and strongly monotone operator, then the above variational inequality problem has a unique solution.Definition 5. A mapping S : H → H is said to be α-inverse strongly monotone (α-ism) if there exists a positive constant Recall that the metric (nearest point) projection from H onto a nonempty closed convex subset C of H, denoted by P C , is defined as follows: for each u ∈ H, Then, P C is characterized by the inequality (for u ∈ H) It is well known that P C and I − P C are firmly nonexpansive and 1-ism.Definition 6.A function h : H → R is said to be weakly lower semi-continuous (w-lsc) at u if u n u implies Lemma 1. [34] Let K be a nonempty closed convex subset of real Hilbert space H. Let {u n } be a sequence which satisfies the following properties: (a) every weak limit point of {u n } lies in K; and (b) lim n→∞ u n − u exists for every u ∈ K.
Then, {u n } converges weakly to a point in K.
Lemma 2. [35] Assume that {s n } is a sequence of nonnegative real numbers such that for each n ≥ 0, where {λ n } is a sequence in (0, 1), {η n } is a sequence of nonnegative real numbers and {δ n } and {µ n } are two sequences in R such that: lim n→∞ µ n = 0; and (c) lim l→∞ η n l = 0 implies lim sup l→∞ δ n l ≤ 0 for any subsequence {n l } ⊂ {n}.

Weak Convergence Theorems
The CQ algorithm in Equation (2) involves two projections P C and P Q and hence might be hard to be implemented in the case where one of them fails to have a closed-form expression.Now, we use the dual variable to propose a new relaxed CQ algorithm for solving the SFP (Equation ( 1)) where the closed convex subsets C and Q are level sets of convex functions.We just need projections onto half-spaces, thus the algorithm is implementable in this case. Let where c : H 1 → R and q : H 2 → R are convex and lower semi-continuous functions.We assume that c and q are subdifferentiable on H 1 and H 2 , respectively.Namely, the subdifferentials, for all v ∈ Q.We also assume that ∂c and ∂q are bounded operators (i.e., bounded on bounded sets).
In this paper, we solve the SFP (Equation ( 1)) under the above assumptions.We note that every convex function defined on a finite-dimensional Hilbert space is subdifferentible and its subdifferential operator is a bounded operator. Set where ξ n ∈ ∂c(u n ), and where η n ∈ ∂q(Au n ).Obviously, C n and Q n are half-spaces and it is easy to verify the C ⊆ C n and Q ⊆ Q n for every n ≥ 0 from the subdifferentiable inequality.
Algorithm 1.Let u 0 , e 0 ∈ H 1 be arbitrary.For n ≥ 1, let where Let {(e n , u n )} be the sequence generated by Algorithm 1, then the sequence {u n } converges weakly to a point u * ∈ Φ and the sequence {(e n , u n )} weakly converges to the point (0, u * ).
Proof.First, we show that lim n→∞ u n − u exists for any u ∈ Φ. Taking u ∈ Φ, we have u ∈ C ⊆ C n and Au ∈ Q ⊆ Q n for all n ∈ N. We know that I − P C n and I − P Q n are 1-ism for all n ∈ N. Thus, from Algorithm 1, we have and Thus, from Equations ( 22) and ( 23), we have Since 2 e n , e n+1 − e n+1 2 = e n 2 − e n+1 − e n 2 , we obtain It follows from ( By Equations ( 25) and ( 27), we obtain ( Let s n = u n − u 2 + λ e n 2 , then the sequence {s n } is lower bounded.By the assumptions on {µ n } and λ, from Equation ( 28) we can get s n+1 ≤ s n , which implies that the sequence {s n } is non-increasing and thus lim n→∞ s n exists.Thus, it follows that {s n } is bounded and hence {u n } is bounded.
Moreover, from Equation ( 28), we also have and lim Thus, lim n→∞ u n − u 2 = lim n→∞ (s n − λ e n 2 ) = lim n→∞ s n exists.Next, we prove ω ω (u n ) ⊆ Φ.From Algorithm 1, we have and Combining Equations ( 29) and ( 30), we get It follows from Algorithm 1 that which implies that Since ∂c is bounded on bounded sets, there exists a constant ξ > 0 such that ξ n ≤ ξ for all n ∈ N. It follows that By Equations ( 30), ( 33) and ( 36), we have Assume that u ∈ ω w (u n ), i.e., there exists a subsequence {u n j } of {u n } such that u n j u as j → ∞.By the weak lower semicontinuity of c and Equation ( 37), we have It follows from Equation (28), the weak lower semicontinuity of q and the fact that Au n j A u that q(A u) ≤ lim inf j→∞ q(Au n j ) ≤ 0. (41) By Lemma 1, we have u n u * and the sequence {(e n , u n )} weakly converges to the point (0, u * ), where u * ∈ Φ.This completes the proof.Remark 1.When λ = 1, Algorithm 1 becomes the relaxed CQ algorithm in Equation ( 16) proposed by Xu [27] for solving the SFP where the closed convex subsets C and Q are level sets of convex functions.Thus, Theorem 1 extends the related results of Xu [27] for solving the SFP (Equation ( 1)).

Strong Convergence Theorems
In this section, we modify the proposed Algorithm 1 to show that the algorithm has strong convergence.It is known that the viscosity approximation method is often used to approximate a fixed point of a nonexpansive mapping U in Hilbert spaces with the strong convergence, which it is defined as follows [37]: for each n ≥ 1, where {β n } ⊆ [0, 1] and h is a contractive mapping.Now, we adapt the viscosity approximation method to get the strong convergence result for solving the SFP (Equation ( 1)) where the closed convex subsets C and Q are given in Equation ( 18).

Algorithm 2.
Let h : H 1 → H 1 be a ρ-contraction mapping and C n and Q n given in Equations ( 19) and (20), respectively.Let u 0 , e 0 ∈ H 1 be arbitrary.For n ≥ 0, let Theorem 2. Assume that {β n }, {µ n }, λ and ρ satisfy the following assumptions: (i) Then, the sequence (e n , u n ) generated by Algorithm 2 strongly converges to (0, u * ), where u * ∈ Φ and u * solves the following variational inequality problem: for any u ∈ Φ.
Proof.Let u * ∈ Φ be unique solution of the variational inequality problem (42).Then, (43) In particular, we have From Algorithm 2, we get Setting s n = u n − u * 2 + λ e n 2 , we have It follows from induction that for each n ≥ 0, which implies that {u n } and {e n } are bounded.In addition, {u n }, {e n }, {h(u n )} and {h(e n )} are bounded.It follows from Algorithm 2 that Thus, by Equations ( 44) and (47), we have where and On the other hand, from Equations ( 43) and (47), we have (50) Now, by setting and Equation (50) can be rewritten in the following form: for each n ≥ 0. By the assumptions on {β n } and ρ, we have To use Lemma 2, it suffices to verify that, for any subsequence{n l } ⊂ {n}, lim l→∞ η n l = 0 implies lim sup Since lim l→∞ η n l = 0, from the assumptions on λ and {µ n }, we obtain lim l→∞ we obtain lim In a similar way to the proof of Theorem 1, we can get to get Equation (52), we only need to verify lim sup for all u = (u 1 , u 2 , u 3 ) T ∈ R 3 and v = (v 1 , v 2 , v 3 ) T ∈ R 3 .
Similar to Example 1, we compared the proposed modified relaxed CQ Algorithm 1 with the relaxed CQ algorithm in Equation (34) proposed by Xu [27] to solve this example.We took µ n = 1.55 and the same stopping criterion as in Example 1.We took different u 0 and e 0 as initial points.The numerical results are given in Tables 3 and 4. We can see in Tables 1-4 that Algorithm 1 was efficient and behaved better than the relaxed CQ algorithm in Equation (34) when choosing a suitable parameter λ for solving Examples 1 and 2.
3 , A = (a i,j ) N×N and a i,j ∈ (0, 1) are generated randomly, the nonemptyclosed convex set C = {u ∈ R 3 | c(u) ≤ 0} and Q = {v ∈ R 3 | q(v) ≤ 0}, where c(u) = −u 1 + u 2 2 + u 2 3and q H is said to be firmly nonexpansive if 2S − I is nonexpansive or, equivalently, u − v, Su − Sv ≥ Su − Sv 2 for all u, v ∈ H.Alternatively, a mapping S : H → H is firmly nonexpansive if and only if S can be expressed as Definition 1.A mapping S : H → H is said to be nonexpansive if Su − Sv ≤ u − v for all u, v ∈ H. Definition 2. A mapping S : H → where I denotes the identity mapping on H and U : H → H is a nonexpansive mapping.Definition 3. A mapping h : H → H is said to be ρ-contraction if there exists a constant ρ∈ [0, 1) such that h(u) − h(v) ≤ ρ u − v for all u, v ∈ H.Definition 4.A mapping h : C → H is said to be η-strongly monotone if there exists a positive constant η such that h