Abstract
Suppose each of is a maximal monotone, and is firmly nonexpansive with . In this paper, we have two purposes: the first is finding the zeros of , and the second is finding the zeros of . To address the first problem, we produce fixed-point equations on the original Hilbert space as well as on the product space and find that these equations associate with crucial operators which are called generalized forward–backward splitting operators. To tackle the second problem, we point out that it can be reduced to a special instance of by defining new operators on the product space. Iterative schemes are given, which produce convergent sequences and these sequences ultimately lead to solutions for the last two problems.
1. Introduction
In the field of image processing and restoration, the simultaneous optimization of multiple objectives is often necessary to achieve desired results. For instance, when restoring historical paintings, balancing original brushstrokes while also removing degradation is crucial. To tackle this complex optimization problem, researchers have proposed various algorithms and techniques.
In what follows, we abstract the above issues into the following two types of problems and proceed to solve them. Let and be two positive numbers, and let be a real Hilbert space with inner product and associated norm . We consider two classes of minimization problems
and
where each is a lower semicontinuous, and proper and convex functions on and g is differentiable with the -Lipschitz gradient. As described in [1], the discussion can be conducted in a general setting where problems (1) and (2) are special instances of those that follow, respectively:
where is the maximal monotone and is firmly nonexpansive. The main purpose of this paper is to present algorithms to solve these inclusions and consider the convergence of the proposed methods.
Finding the zeros of an operator is a crucial task in operator theory [1,2]. A number of ideas have been proposed to address the issue of when the operator is decomposed into a sum of two. For instance, Ref. [3] developed the forward–backward method to solve when one of the operator is the maximal monotone and the other is firmly nonexpansive. We remark that by this, we gain a solution (3) in the case . On the other hand, in the case of when each of the two operators is maximal monotone, the authors of [4] presented the Douglas–Rachford method, while the authors of [5] developed the splitting operator. Likewise, we remark that it happens to be the special case of problem (4) for the case of Details of these two algorithms can be found in Section 2.2.
It is rather difficult to extend the forward–backward method to address problem (3) for a general n. Nevertheless, in [6], the authors achieved an algorithm. This proves to be covered by the four kinds of generalized forward–backward method given in this paper.
As for problem (4), extension variants of the Douglas–Rachford method have emerged. Most of them rely on reducing the original problem to the simple case of by introducing auxiliary variables. In this paper, we point out that this problem can be solved by applying the generalized forward–backward method described above with . Additionally, we prove that this method can also be explained as a reduction in the original problem for the case . A benefit of this observation is that we can design splitting operators which allow us to solve (4) by the proximal point algorithm. The work of [7] is also included in this paper. For recent improvements in this field, please refer to references [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24].
This paper is divided into five sections. The conception of the product space as well as some of the operators on it are introduced in Section 2. To address the general problem, we will review in this section some classical methods for finding the zeros of a sum of two operators. Section 3 is devoted to solving problem (3) by deriving fixed-point equations. In this process, we obtain the so called generalized forward–backward splitting methods. In Section 4, we solve problem (4) from two different perspectives. Firstly, we view it as an instance of (3), so we obtain fixed-point equations as a consequence of the generalized forward–backward methods. Secondly, we point out by introducing new operators on the product space that this problem can be reduced to the case of , which has been solved in Section 2. Based on this, we can obtain fixed-point equations in another way and we can generate splitting operators with good properties. We provide in Section 5 iterative schemes which produce convergent sequences, and these sequences eventually lead to solutions of either problem (3) or (4).
2. Foundations
In this section, we shall introduce some notations to elaborate problems (3) and (4). Also, existing work which can be used to address these two problems for a special n will be reviewed.
2.1. Notations and Definitions
We begin by introducing some notations and properties of operators on .
Definition 1
([1]). Let A be a set-valued operator , where the graph of A is defined as the set . No distinctions between an operator and its graph are made, that is, denotes . Its inverse is defined as The zeros of A is defined as
The domain and range of A are, respectively,
Definition 2
([1]). An operator is single-valued if and the cardinality of is at most 1.
Definition 3
([1]). If are two set-valued operators on , the resolvent and the reflection operator are defined, respectively, as
Throughout this paper, let denote the identity operator on the appropriate Hilbert space.
Definition 4
([1]). An operator is monotone if Moreover, A is maximal monotone if its graph is not strictly contained in any other monotone operator.
Definition 5
([1]). An operator is nonexpansive if
Let An operator A is α-averaged nonexpansive if there exists a nonexpansive operator R such that . In this case, we use the notation .
Definition 6
([1]). An operator is firmly nonexpansive if
Furthermore, A is firmly nonexpansive if and only if .
Lemma 1
([1]). Let be a monotone operator, then it is maximal monotone if and only if
Lemma 2
([1]). Let be such that set , and . Then, A is maximal monotone if and only if T is firmly nonexpansive and .
2.2. Product Space
Let be positive constants such that , and we define the product space endowed with the following scalar product and the corresponding norm. Below are some elementary operators in the product space . Let K be a subspace of , and , be its orthogonal complementary, the normal cone operator, respectively:
Moreover, let be operators given in (3), whereby two operators and on are defined as follows: for
It is easy to establish that is maximal monotone, and is thus single-valued.
Lemma 3
([1]). Define the canonical isometry as Then, for any , the following statements hold:
- (i)
- ,;
- (ii)
- Let then .
- (iii)
- .
Lemma 4.
Let be elements in , then
Proof.
This relation can be verified by applying a weighted sum on both sides of the equations. □
Below is an equivalent form of the last lemma in the product space.
Lemma 5.
Let , then if and only if .
2.3. Zeros of a Sum of Two Operators
In this subsection, we will review some classical methods to find the zeros of a sum of two operators, i.e.,
When A is maximal monotone and B is firmly nonexpansive, we find that the inclusion (7) is equivalent to the next fixed-point equation. The corresponding operator T is called the forward–backward splitting operator.
Lemma 6.
Let A be maximal monotone and B be firmly nonexpansive. Define
then
When A and B are both maximal monotone, we propose two fixed-point equations to solve problem (7).
Lemma 7.
Let A and B be maximal monotone, then if and only if and z is a fixed point of the following firmly nonexpansive operator .
Lemma 8.
Let A and B be maximal monotone, then if and only if and z is a fixed point of the following firmly nonexpansive operator ,
The operator is called the Douglas–Rachford splitting operator and it was presented by Lions and Mercier [4]. It is well known that is firmly nonexpansive; thus, Lemma 2 says that there exists a maximal monotone operator such that . In [5], Jonathan Eckstein constructed his splitting operator as follows:
3. Generalized Forward–Backward Methods
In this section, we consider the problem (3). Recall that in Proposition 6, we dealt with this problem for by deriving a fixed-point equation. We shall develop variant fixed-point equations and algorithms for general n.
3.1. Fixed-Point Equations on
Proposition 1.
Let , then if and only if there exists such that
Proof.
By definition, we have ⇔ ⇔ In addition, if and only if . Thus, we obtain the equivalent condition (12) by denoting . In the above and later discussions, note that B is a single-valued operator which can be established from . □
As a direct consequence of this proposition, we have the following result.
Proposition 2.
Let , then if and only if there exists such that
Proof.
Let in Equation (12). □
Proposition 3.
Let , then if and only if there exists such that
Proof.
Let in Equation (12). □
Proposition 4.
Let , then if and only if there exists such that
Proof.
Let in Equation (12). □
3.2. Fixed-Point Equations on the Product Space
In this subsection, the fixed-point Equations (12)–(15) will be further studied. Firstly, we define four operators on the product space .
Definition 7.
Let . Define four operators as follows:
where and
Analogous to the forward–backward operator T in (8), these four operators are called generalized forward–backward splitting operators. We propose below the new fixed-point equations in the product space.
Proposition 5.
Let , then if and only if there exists some such that any of the following equations holds:
- (i)
- (ii)
- (iii)
- (iv)
3.3. Characterization of Forward–Backward Splitting Operators
Proposition 6.
Let be defined in Definition 7; then, we have
Proof.
(i) Let with then, we can check that
Moreover, definition (16) can be written equivalently as
These results lead to
or equivalently
By noting (34), we can obtain relation (30).
- (ii)
- Let with then, we can check thatThus, we can deduce from (17) that i.e.,Combining this with the fact that , we can verify the equality (31).
- (iii)
- Let with then we can check that Furthermore, we haveor equivalently,So, it can be induced from definition (18) thatand thus,
- (iv)
- It is readily induced from (19) thatLet with then, we can check that andIn other words, we gain
□
Lemma 9
([1]). For any we have
Proposition 7.
Let be defined in Definition 7. For any define then each is α-averaged nonexpansive.
Proof.
It follows from [6], Proposition 4.6. □
Note that a fixed-point equation for problem (3) was also proposed in [6]. We remark that the associated operator in that context is nothing but in our work; thus, we have covered the method given by [6].
4. Zeros of a Sum of n Maximal Monotone Operators
The main focus of this section is to present fixed-point equations as well as algorithms to solve problem (4).
4.1. An Application of the Generalized Forward–Backward Methods
Observe that it can be identified as a special instance of problem (3) with , so we can apply the methods given in Proposition 5 to obtain a solution. In this case, the operators are simplified to P, and to Q, respectively:
The following results are a direct consequence of Lemma 9 and Proposition 5.
Proposition 8.
Let , then if and only if there exists such that is a fixed point of P or Q and .
Proposition 9.
The operators P and Q are both firmly nonexpansive.
We also present some properties of the fixed points of the operators P and Q.
Lemma 10.
Let be a fixed point of the operator P, then we have
Proof.
It suffices to prove that . By assumption, we have . So, we obtain that thus □
Lemma 11.
Let be a fixed point of the operator Q, then we have
Proof.
It suffices to prove that . By assumption, we have . It can be induced from the linearity of the operator that , and thus, So, we obtain that Note that the first and second identities hold due two Proposition 5 and (ii) of Proposition 3, respectively. □
It seems that we have solved the problem perfectly. Nevertheless, we shall investigate what the intrinsic mode is. This research is conducted from a distinctive perspective.
4.2. Reformulation of Fixed-Point Equations
We shall begin with a crucial proposition which reveals that finding the zero of the sum of n maximal monotone operators can be reduced to the simple case of .
Proposition 10.
Let then if and only if with .
Proof.
We investigate problem (4) as follows:
Define then we obtain an equivalent relation
□
Based on the above discussions, we can regain Proposition 8 by an alternative approach.
Proposition 11.
Let , then if and only if there exists such that is a fixed point of P and .
Proof.
Let then by Lemma 7, is equivalent to
Note that the second identity states that , and thus, we conclude from Lemma 10 that . □
Proposition 12.
Let , then if and only if there exists such that is a fixed point of Q and .
Proof.
Similar to the proof of the last proposition, let then, by Lemma 7, is equivalent to
The second identity states that . It follows from Lemma 11 that . And, we conclude from Lemma 10 that . □
Therefore, although the above fixed-point methods originate from applications of generalized forward–backward methods, they intrinsically rely on bringing back the original problem to the case of . As previously discussed, most methods can be explained as extensions of the Douglas–Rachford splitting. We remark that our work is an extension of algorithms described in Lemma 7.
4.3. Splitting Operators
In this subsection, we will design splitting operators for problem (4) with general n. It is clear from Lemma 10 that we only have to investigate the next one:
Inspired by construction (11), we can generate a splitting operator as
Note that here, and
Write the identity as This, combined with identity (53), implies that
Therefore, the operator in (52) can be written equivalently as
With some changes in notations, we obtain the next splitting operator.
Definition 8.
Let be any maximal monotone operator, and define
Proposition 13.
Let , then
Proof.
Proposition 14.
Let Q be given in (45), then .
Proof.
By definition, we obtain
It is implied from the second identity of (60) that . Substituting this into the third identity of (60), we obtain that
so
This is equivalent to
Thus, we have , which completes this proof. □
Proposition 15.
is a maximal monotone operator.
Proof.
Assume that are arbitrary elements in . That is,
Let Then
This implies that
Thus,
Note that the last inequality holds due to the fact that and each operator is monotone. Thus, we have proved that is a monotone operator. By Proposition 1, to prove that it is also maximal, it remains to show that . This is quite easy since we have proved in Proposition 14 that is firmly nonexpansive. □
By exchanging the two operators and in (52) and applying the same methods, we obtain a splitting operator , which possesses properties similar to .
Definition 9.
Let be any maximal monotone operator, and then define
Proposition 16.
Let , then
Proposition 17.
Let P be given in (45), then .
Proposition 18.
is a maximal monotone operator.
Recall that we have proved in Proposition 9 that are both firmly nonexpansive. In this subsection, we propose constructive approaches to obtain maximal monotone operators and such that
5. Iterative Algorithms and Convergence
The main focus of this section is to present iterative schemes which produce convergent sequences leading to a solution of problem (3) or (4). Firstly, we recall three lemmas which can be used to find , the set of fixed points of an averaged nonexpansive operator A.
Lemma 12.
Let let A be a k-averaged nonexpansive operator on such that and let be a sequence in such that . Set , and define for ,
then, converges weakly to a point in Fix(A).
Under particular circumstances, we can use Picard iteration and obtain sequences which converge to a fixed point of A.
Lemma 13.
Let and let C be a closed and convex set of . If is a k-averaged nonexpansive operator on such that set , and define for ,
then converges weakly to a point in Fix(A).
Lemma 14.
Let A be a k-averaged nonexpansive operator on such that set , and define for ,
then converges weakly to a point in Fix(A).
Based on the above discussions, we present the following series of propositions. We remark that the corresponding iterative schemes lead to a solution of problem (3) or (4). Therefore, we have developed efficient algorithms to solve the minimization problem (1) and (2).
Proposition 19.
Suppose we have and Let be a sequence in such that . For each set , and define for ,
then, the following hold:
- (i)
- For each there exists , such that .
- (ii)
- For we have
- (iii)
- For we have
Proof.
This relation is due to Propositions 5 and 7 and Lemma 12. □
Similarly, Propositions 5 and 7 and Lemma 13 result in the next result.
Proposition 20.
Suppose we have and For each set , and define for ,
then, the following hold:
- (i)
- For each there exists , such that .
- (ii)
- For we have
- (iii)
- For we have
Proposition 21.
Suppose we have and Set , and define
then, the following hold:
- (i)
- There exists , such that .
- (ii)
Proof.
This relation follows from Propositions 8 and 9 and Lemma 14. □
6. Conclusions
In this paper, we have explored iterative schemes for solving problems (3) or (4) and introduced the generalized forward–backward splitting methods along with the introduction of new operators. By introducing new operators, we were able to simplify the problems to the case of n = 2 and obtain fixed-point equations and splitting operators with desirable properties in a different way.
Our research has shown that the generalized forward–backward splitting methods are highly beneficial. Through these methods, we can generate convergent sequences that eventually lead to the solution of the problems.
Additionally, the introduction of new operators has proved to be effective in simplifying the problems and enabling a better understanding and handling of complex situations. When the zeros of the sum of n operators are reduced to the zeros of just two operators, symmetry manifests in the potential alignment of properties and patterns across the operators. If the operators possess symmetry, such as invariance under certain transformations, the zeros of these operators may exhibit similar behaviors or locations. This symmetry allows us to generalize and simplify the problem by focusing on just two representative operators.
Author Contributions
All authors contributed equally to this paper: Conceptualization, H.X.; validation, Z.L. and Y.Z.; data curation, X.L.; writing—original draft preparation, H.X.; writing—review and editing, X.L. All authors have read and agreed to the published version of the manuscript.
Funding
This work was supported by the specialized research fund of Yibin University (Grant No. 412-2021QH027).
Data Availability Statement
The authors confirm that the data supporting the findings of this study are available within the article.
Acknowledgments
The authors would like to express special thanks to Wang Jinrong and Qiu Xiaoling for their valuable suggestions during the process of writing. All authors contribute equally to this work.
Conflicts of Interest
The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
References
- Bauschke, H.H.; Combettes, P.L. Convex Analysis and Monotone Operator Theory in Hilbert Space; Springer: Berlin/Heidelberg, Germany, 2010. [Google Scholar]
- Minty, G.J. Monotone (Nonlinear) Operators in Hilbert space. Duke Math. J. 1962, 29, 341–346. [Google Scholar] [CrossRef]
- Chen, G.H.-G.; Rockafellar, R.T. Convergence rates in forward-backward splitting. Siam J. Optim. 1997, 7, 421–444. [Google Scholar] [CrossRef]
- Douglas, J.; Rachford, H.H. On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 1956, 82, 421–439. [Google Scholar] [CrossRef]
- Eckstein, J. The Lions-Mercier Splitting Algorithm and the Alternating Direction Method Are Instances of the Proximal Point Algorithm; Laboratory for Information and Decision Systems: Cambridge, MA, USA, 1988. [Google Scholar]
- Raguet, H.; Fadili, J.; Peyre, G. Generalized Forward-Backward Splitting. SIAM J. Imaging Sci. 2013, 6, 1199–1226. [Google Scholar] [CrossRef]
- Xiao, H.; Zeng, X. A proximal point method for the sum of maximal monotone operators. Math. Methods Appl. Sci. 2014, 37, 2638–2650. [Google Scholar] [CrossRef]
- Camlibel, K.; Iannelli, L.; Tanwani, A. Convergence of proximal solutions for evolution inclusions with time-dependent maximal monotone operators. Math. Program 2022, 194, 1017–1059. [Google Scholar] [CrossRef]
- Dadashi, V.; Postolache, M. Forward–backward splitting algorithm for fixed point problems and zeros of the sum of monotone operators. Arab. J. Math. 2020, 9, 89–99. [Google Scholar] [CrossRef]
- Boikanyo, O.A. The viscosity approximation forward-backward splitting method for zeros of the sum of monotone operators. Abstr. Appl. Anal. 2016, 2016, 2371857. [Google Scholar] [CrossRef][Green Version]
- Dadashi, V. On a hybrid proximal point algorithm in Banach spaces. Univ. Politeh. Buchar. Ser. A 2018, 80, 45–54. [Google Scholar]
- Dadashi, V.; Khatibzadeh, H. On the weak and strong convergence of the proximal point algorithm in reflexive Banach spaces. Optimization 2017, 66, 1487–1494. [Google Scholar] [CrossRef]
- Dadashi, V.; Postolache, M. Hybrid proximal point algorithm and applications to equilibrium problems and convex programming. J. Optim. Theory Appl. 2017, 174, 518–529. [Google Scholar] [CrossRef]
- Moudafi, A. On the convergence of the forward-backward algorithm for null-point problems. J. Nonlinear Var. Anal. 2018, 2, 263–268. [Google Scholar]
- Yuan, H. A splitting algorithm in a uniformly convex and 2-uniformly smooth Banach space. J. Nonlinear Funct. Anal. 2018, 26, 1–2. [Google Scholar] [CrossRef]
- Combettes, P.L. Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization 2004, 53, 475–504. [Google Scholar] [CrossRef]
- Combettes, P.L.; Pesquet, J.-C. A Douglas-Rachford splitting approach to nonsmooth convex variational signal recovery. IEEE J. Sel. Top. Signal Process. 2007, 1, 564–574. [Google Scholar] [CrossRef]
- Combettes, P.L.; Pesquet, J.-C. A proximal decomposition method for solving convex variational inverse problems. Inverse Problems 2008, 24, 65014–65040. [Google Scholar] [CrossRef]
- Eckstein, J. Parallel alternating direction multiplier decompositon of convex programme. J. Optim. Theory Appl. 1994, 80, 39–62. [Google Scholar] [CrossRef]
- Eckstein, J.; Svaiter, B.F. General projective splitting methods for sums of maximal monotone operators. SIAM J. Control Optim. 2009, 48, 787–811. [Google Scholar] [CrossRef]
- Minty, G.J. On the maximal domain of a “monotone” function. Mich. Math. J. 1961, 8, 135–137. [Google Scholar] [CrossRef]
- Rockafellar, R.T. On the maximality of Sums of Nonlinear Monotone Operators. Trans. Am. Math. Soc. 1970, 149, 75–88. [Google Scholar] [CrossRef]
- Rockafellar, R.T. Monotone operators and the proximal point algorithm. SIAM J. Control. Optim. 1976, 14, 877–898. [Google Scholar] [CrossRef]
- Tseng, P. A modified forward-backward splitting method for maximal monotone mappings. SIAM J. Control Optim. 2000, 38, 431–446. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).