The Combination Projection Method for Solving Convex Feasibility Problems

In this paper, we propose a new method, which is called the combination projection method (CPM), for solving the convex feasibility problem (CFP) of finding some x∗ ∈ C := ∩m i=1{x ∈ H | ci(x) ≤ 0}, where m is a positive integer, H is a real Hilbert space, and {ci} i=1 are convex functions defined as H. The key of the CPM is that, for the current iterate xk, the CPM firstly constructs a new level set Hk through a convex combination of some of {ci} i=1 in an appropriate way, and then updates the new iterate xk+1 only by using the projection PHk . We also introduce the combination relaxation projection methods (CRPM) to project onto half-spaces to make CPM easily implementable. The simplicity and easy implementation are two advantages of our methods since only one projection is used in each iteration and the projections are also easy to calculate. The weak convergence theorems are proved and the numerical results show the advantages of our methods.


Introduction
Let H be a real Hilbert space with inner product •, • and norm • , respectively.Recall that the projection operator of a nonempty closed convex subset D of H, P D : H → D, is defined by It is inevitable to use projections to solve convex feasibility problems (CFP) [1][2][3] split feasibility problems (SFP) [4,5], and variational inequality problems (VIP) [6,7].
If the set D is simple, such as a hyperplane or a halfspace, the projection onto D can be calculated explicitly.However, it is well known that in general, D is very complex, and P D has no closed form formula, for which, the computation of P D is rather difficult [8].So, how to efficiently compute P D is a very important and interesting problem.Fukushima [9] suggested the half-space relaxation projection method and the idea was followed by many authors to introduce relaxed projection algorithms for solving the SFP [10,11] and the VIP [12,13].
Let m be a positive integer and {C i } m i=1 be a finite family of nonempty closed convex subsets of H with a nonempty intersection.The convex feasibility problem [14] is to find which is very common problem in diverse areas of mathematics and physical sciences [15].In the last twenty years, there has been growing interests in the CFP since it was found to have various applications in imaging science [16,17], medical treatments [18], and statistics [19].
A great deal of literature on methods for solving the CFP have been published (e.g., [20][21][22][23]).The classical method traces back at least to the alternating projection method introduced by von Neumann [14] in 1930s, which is called the successive orthogonal projection method (SOP) in Reference [24].The SOP in Reference [14] solves the CFP with C 1 and C 2 being two closed subspaces in H, and generates a sequence {x k } ∞ k=1 via the iterating process: where x 0 ∈ H is an arbitrary initial guess.Von Neumann [14] proved that the sequence {x k } ∞ k=1 converges strongly to P C 1 ∩C 2 x 0 .In 1965, Bregman [2] extended von Neumann's results to the case where C 1 and C 2 are closed convex subsets and proved the weak convergence.Hundal [25] showed that for two closed convex subsets C 1 and C 2 , the SOP does not always converge in norm by providing an explicit counterexample.Further results on the SOP were obtained by Gubin et al. [26] and Bruck et al. [27].
The SOP is the most fundamental method to solve CFP, and many existing algorithms [24,28] can be regarded as generalizations or variants of the SOP.Let {C i } m i=1 be a finite family of level sets of convex functions {c i } m i=1 (i.e., [9], He et al. [28] introduced a contraction type sequential projection algorithm which generates the iterating process: where the sequence {λ k } ∞ k=0 ⊂ (0, 1), u ∈ H is a given point and {C i k } m i=1 is a finite family of half-spaces such that C i k ⊃ C i for i = 1, 2, • • • , m and all k ≥ 0. They proved that the sequence {x k } ∞ k=1 converges strongly to P C u under certain conditions.Because the projection operators onto half-spaces have closed-form formulae, the algorithm (3) seems to be easily implemented.However, one common feature of SOP-type algorithms is that they need to evaluate all the projections {P C i } m i=1 (or relaxed projections {P C i k } m i=1 ) in each iteration (see, e.g., Reference [24]), which results in prohibitive computational cost for large scale problems.
Therefore, to solve the CFP (1) efficiently, it is necessary to design methods which use fewer projections in each iteration.He et al. [29,30] proposed the selective projection method (SPM) for solving the CFP (1) where C is the intersection of a finite family of level sets of convex functions.An advantage of the SPM is that we only need to compute one (appropriately selected) projection in each iteration, and the weak convergence of the algorithm is still guaranteed.More precisely, the SPM consists of two steps in each iteration.Step one, once the k-th iterate x k is obtained, according to a certain criterion, we select one set Step two, we then update the new iterate x k+1 via the process: Because (4) only involves one projection, the SPM is simpler than the SOP-type algorithms.
The main purpose of this paper is to propose a new method, which is called the combination projection method (CPM), for solving the convex feasibility problem of finding some where m is a positive integer and {c i } m i=1 are convex functions defined as H.The key of the CPM is that for the current iterate x k , the CPM firstly constructs a new level set H k through a convex combination of some of {c i } m i=1 , and then updates the new iterate x k+1 by using the projection P H k .The simplicity and ease of implementation are two of the advantages of our method since only one projection is used in each iteration and the projections are easy to calculate.To make the CPM easily implementable, we also introduce the combination relaxation projection method (CRPM), which involves projection onto half-spaces.The weak convergence theorems are proved and the numerical results show the advantages of our methods.In fact, the methods in this paper can be easily extended to solve other nonlinear problems, for example, the SFP and the VIP.

Preliminaries
Let H be a real Hilbert space and T : H → H be a mapping.Recall that H → H is an averaged mapping if there exists some α ∈ (0, 1) and nonexpansive mapping V : H → H such that T = (1 − α)I + αV; in this case, T is also said to be α-averaged.
In this case, we say that T is ν-ISM.

Lemma 1 ([31]
).For a mapping T : H → H, the following are equivalent: Recall that the projection onto a closed convex subset D of H is defined by It is well known that P D is characterized by the inequality Some useful properties of the projection operators are collected in the lemma below.

Lemma 3 ([32]
).Let D be a nonempty closed convex subset of H. Let {u k } ∞ k=0 ⊂ H satisfy the properties: Then, {u k } ∞ k=0 converges weakly to a point in D.

Lemma 4.
Let {c i } m i=1 be a finite family of convex functions defined as H such that their level sets Then, the following properties are satisfied.
is a closed ball or a half-space, and at least one of them is a closed ball.

Proof. (i) Obviously, for any {β
Since {v i } m i=1 is a linearly independent group, we assert that ∑ m i=1 β i v i = 0, and hence, it is easy to see from ( 6) that H is a half-space. (ii We directly deduce Consequently, (iii) Assume that C 1 is a closed ball and C 2 is a half-space, then c 1 and c 2 have the forms c 1 (x By using the same argument as in (ii), we assert and this means that H is indeed a closed ball.This together with (i) and (ii) indicates that the conclusion is true for the general case.
We denote by ∂ f (x) the set of all subgradients of f at x. Recall that f is said to be subdifferentiable at x if ∂ f (x) = ∅, and f is said to be subdifferentiable (on H) if it is subdifferentiable at every x ∈ H. Recall also that the inequality ( 9) is called the subdifferential inequality of f at x.

The CPM for Solving Convex Feasibility Problems
In this section, the combination projection method (CPM) is proposed for solving the convex feasibility problem (CFP): where The algorithm proposed below for solving the CFP ( 10) is called the combination projection method (CPM) for the reason that the projection that is used to update the next iterate is on the level set of a convex combination of some of {c i } m i=1 in an appropriate way.Throughout this section, we always assume that C = ∅ and use I to represent the index set {1, 2, • • • , m} for convenience.
Remark 1. Algorithm 1 suits for the case where {P H k } ∞ k=0 have closed-form representations.For example, according to Lemma 4, if each of {C i } m i=1 is a closed ball or a half-space, then H k is also a closed ball or a half-space for each k ≥ 0, and hence, P H k has the closed-form representation for all k ≥ 0. In this case, Algorithm 1 is easy implementable.
Step 2: Given the current iterate x k .Check the index set i.e., c i (x k ) ≤ 0 for all i = 1, 2, • • • , m, then stop and x k is a solution of the CFP (10).Otherwise, select {β , and construct the level set: Step 3: Compute the new iterate x k+1 := P H k (x k ).
Set k := k + 1 and return to Step 2.
Remark 2. The simplicity and ease of implementation of Algorithm 1 can be illustrated through a simple example in R 3 .Compute the projection P C u, where u = (0, 0, 4) ∈ R 3 and C ⊂ R 3 is given by Selecting the initial guess x 0 = u and using the CPM (Algorithm 1), only one iteration step is needed to get the exact solution of the problem.Indeed, since c i (0, 0, 4) > 0 for each i = 1, 2, 3, 4, then I 0 = {1, 2, 3, 4}.Taking the convex combination coefficients as β i = 1 4 , i = 1, 2, 3, 4, the CPM firstly generates a new set e., the level set of the convex function , then, the CPM updates the new iterate x 1 = P H 0 x 0 = (0, 0, 1) = P C u.However, if we adopt the SOP to get P C u, the iteration process will be complicated.On one hand, although there is an expression for the projection onto an ellipsoid [8], obtaining a constant in the expression requires solving an algebraic equation.On the other hand, the actual calculation shows that after several iterations, we can only get an approximate solution of P C u.
We have the following convergence result for Algorithm 1.
Theorem 1. Assume that for each i = 1, 2, • • • , m, c i : H → R is a bounded uniformly continuous (i.e., uniformly continuous on each bounded subset of H) and convex function.
k=0 generated by Algorithm 1 converges weakly to a solution of CFP (10).
Proof.Obviously, we may assume that x k / ∈ C for all k ≥ 0 with no loss of generality.By the definition of H k , it is very easy to see that C ⊂ H k holds for all k ≥ 0. For any x * ∈ C, we have by Lemma 2 that From ( 11), we assert that { x k − x * } is nonincreasing; hence, {x k } ∞ k=0 is bounded and lim k→∞ x k − x * 2 exists.Furthermore, we also get Particularly, x k − x k+1 → 0 as k → ∞.By Lemma 3, all we need to prove is that ω w (x k ) ⊂ C. To see this, take x ∈ ω w (x k ) and let {x k j } ∞ j=1 be a subsequence of {x k } ∞ k=0 weakly converging to x. Noticing x k+1 ∈ H k , we get For each fixed i ∈ I and any j ≥ 0, if i / ∈ I k j , then If i ∈ I k j , by virtue of the definition of I k j and ( 12), we get Moreover, noting the combination of ( 13) and ( 14) yields for each i = 1, 2, • • • , m and all j ≥ 0. Since x k j x, x k j − x k j +1 → 0, and {c l } m l=1 are w-lsc and bounded uniformly continuous, we can obtain c i ( x) ≤ 0 by taking the limit in (15) as j → ∞.Hence x ∈ C and ω w (x k ) ⊂ C.This completes the proof.
The second algorithm for solving the CFP ( 10) is named the combination relaxation projection method (CRPM), which works for the case where the projection operators {P H k } ∞ k=0 do not have closed-form formulae.In this case, we assume that the convex functions {c i } m i=1 are subdifferentiable on H.The convergence of Algorithm 2 is given as follows.
Step 2: Given the current iterate x k .Check the index set i.e., c i (x k ) ≤ 0 for all i = 1, 2, • • • , m, then stop and x k is a solution of the CFP (10).Otherwise, select {β and construct a half space by where Set k := k + 1 and return to Step 2.

Theorem 2. Assume that for each
, convex function such that the subdifferential mapping ∂c i is bounded (i.e., bounded on bounded subsets of H).
then the sequence {x k } ∞ k=0 generated by Algorithm 2 converges weakly to a solution of the CFP (10).
Proof.With no loss of generality, we assume x k / ∈ C for all k ≥ 0. First of all, we show that H R k is a half-space, i.e., ∑ i∈I k β (k) i ξ i k = 0. Indeed, if otherwise, it can be asserted by ( 16) that c i (x k ) ≤ 0 holds for each i ∈ I k and this is contradictory to the definition of I k .We next show C ⊂ H R k .Indeed, for each x ∈ C, we have from the subdifferential inequality (9) that where By the definition of H R k (see ( 16)), we assert from (17 This implies that {x k } ∞ k=0 is bounded, lim k→∞ x k − x * 2 exists, and lim k→∞ x k+1 − x k = 0. Now we verify that operator, there exists a constant M ≥ 0 such that ξ i k ≤ M for all k ≥ 0 and i ∈ I k .By the definition of H R k and the fact that For each i ∈ I and k ≥ 0, if i / ∈ I k , then and if i ∈ I k , it follows from the definition of H R k and (18) that Hence, for each i ∈ I, the combination of ( 19) and ( 20) leads to From ( 21), the containment ω w (x k ) ⊂ C follows immediately from an argument similar to the final part of the proof of Theorem 1.

Numerical Results
In this section, we compare the behavior of the CPM (Algorithm 1) and SOP by solving two synthetic examples in the Euclidean space R n .All the codes were written by Matlab R2010a and all the numerical experiments were conducted on a HP Pavilion notebook with Intel(R) Core(TM) i5-3230M CPU@2.60 GHz and 4 GB RAM running on Windows 7 Home Premium operating system.
Obviously, 0 ∈ C, i.e., problem (22) is solvable.We use x k = (x k 1 , x k 2 , . . ., x k n ) to denote the k-th iterate and define Err k := max 1≤i≤m c i (x k ) (23) to measure the error of the k-th iteration, which also serves as the role of checking whether or not the proposed algorithm converges to a solution.In fact, it is easy to see that if Err k is less than or equal to zero, then x k is an exact solution of Problem (22) and the iteration can be terminated; if Err k is greater than zero, then x k is just an approximate solution and the smaller Err k , the smaller the error of x k to a solution.
Let |I k | denote the number of elements of the set I k .We give two ways to choose β (k) i .
(1) β Denote the corresponding combination projection method by CPM2.Table 1 illustrates that the set I k is generally different each iteration.From Figure 1, we conclude that the behaviors of the CPM1 and CPM2 depend on the initial point x 0 .The errors for CPM1 and CPM2 oscillate which may be because only partial information about the convex sets {C i } m i=1 is used in each iteration.However, in view of the SOP, all the information about the convex sets {C i } m i=1 is used in each iteration since it involves all projections {P C i } m i=1 in each iteration.From Figure 2, the CPM1 behaves better than the SOP.Example 2. Consider the linear equation system: where A is an m × n matrix, m < n, and b is a vector in R m .If the noise is taken into consideration, Problem ( 24) where > 0 measures the level of errors.
where i ≥ 0. It is easy to show that Problem (25) is equal to the convex feasibility problem: The set C is nonempty since the linear equation system has infinite solutions.Set Err k := max 1≤i≤m The initial point x 0 is randomly chosen in (0, 10) n .We compared the CPM2 and SOP for different m and n.From Figure 3, the behavior of the CPM2 is better than that of the SOP.The error of the CPM2 has a bigger oscillation than that in Example 2, the oscillation seems to decrease when the iteration is very big.The SOP behaves well when m and n are small, while its error is very big for big m and n.In Figure 4, we compare the CPU time of the CPM2 and SOP, which illustrates that the CPU time of the CPM2 is less than that of the SOP.Furthermore, the CPU time of the SOP exceeds that of the CPM2 with the iteration.

Conclusions
In this paper, we propose the combination projection method (CPM) for solving the convex feasibility problem (CFP).The CPM is simple and easy to implement, and has a fast convergence speed.How to further speed up the convergence for the CPM through selecting the convex combination coefficients {β (k) i } i∈I k in Algorithms 1 and 2 is worthy of further study.

Figure 1 .
Figure 1.The comparison of two choices of β (k) i for different random choices of x 0 for Example 1.(a) Number of iteration.

Figure 2 .
Figure 2. The comparison of the CPM and the successive orthogonal projection method (SOP) for Example 1.

Table 1 .
Comparison of I k of the combination projection method (CPM) 1 and CPM2.