Convergence Analysis of an Inexact ThreeOperator Splitting Algorithm
Abstract
:1. Introduction
Algorithm 1: An inexact threeoperator splitting algorithm 
Input: For arbitrary ${z}^{0}\in H$, choose $\gamma $ and ${\lambda}_{k}$. 
For each $k=0,1,2,\cdots $, compute

Stop when a given stopping criterion is met. 
Output:${x}_{B}^{k},{x}_{A}^{k}$ and ${z}^{k+1}$. 
2. Preliminaries
 (1)
 The set of zeros of A is $zer\phantom{\rule{0.166667em}{0ex}}A:=\{x\in H:0\in Ax\}$;
 (2)
 The domain of A is $dom\phantom{\rule{0.166667em}{0ex}}A:=\{x\in H:Ax\ne \varnothing \}$;
 (3)
 The range of A is $ran\phantom{\rule{0.166667em}{0ex}}A:=\{y\in H:\exists x\in H:y\in Ax\}$;
 (4)
 The graph of A is $gra\phantom{\rule{0.166667em}{0ex}}A:=\left\{\right(x,y)\in H\times H:y\in Ax\}$;
 (5)
 The resolvent of A is ${J}_{A}={(I+A)}^{1}$.
 (1)
 T is firmly nonexpansive.
 (2)
 $2TId$ is nonexpansive.
 (3)
 For all $x,y\in H$, ${\parallel TxTy\parallel}^{2}\le \langle TxTy,xy\rangle $.
 (1)
 T is αaveraged.
 (2)
 $(1\frac{1}{\alpha})Id+\left(\frac{1}{\alpha}\right)T$ is nonexpansive.
 (3)
 For all $x,y\in H$, ${\parallel TxTy\parallel}^{2}\le {\parallel xy\parallel}^{2}\frac{1\alpha}{\alpha}{\parallel (IdT)x(IdT)y\parallel}^{2}$.
 (1)
 $\left\{{x}_{n}\right\}$ is Fejér monotone with respect to $Fix\left(T\right)$.
 (2)
 $\{T{x}_{n}{x}_{n}\}$ converges strongly to 0.
 (3)
 $\left\{{x}_{n}\right\}$ converges weakly to a point in $Fix\left(T\right)$.
 (1)
 $\left\{{x}_{n}\right\}$ is Fejér monotone with respect to $Fix\left(T\right)$.
 (2)
 $\{T{x}_{n}{x}_{n}\}$ converges strongly to 0.
 (3)
 $\left\{{x}_{n}\right\}$ converges weakly to a point in $Fix\left(T\right)$.
3. An Inexact ThreeOperator Splitting Algorithm
 (1)
 $\left\{{z}^{k}\right\}$ is Fejérmonotone with respect to $Fix\left(T\right)$.
 (2)
 $\{T{z}^{k}{z}^{k}\}$ converges strongly to zero.
 (3)
 $\left\{{z}^{k}\right\}$ converges weakly to a fixed point of T.
 (4)
 If ${x}^{*}\in zer(A+B+C)$, then there exists a constant $M>0$ such that, for any ${\lambda}_{k}\in (0,\frac{1}{\alpha})$,$$\sum _{k=0}^{+\infty}{\lambda}_{k}{\parallel C{J}_{\gamma B}{z}^{k}C{x}^{*}\parallel}^{2}\le \frac{1}{\gamma (2\beta \frac{\gamma}{\epsilon})}\left(\parallel {z}^{\circ}{z}^{*}{\parallel}^{2}+\sum _{k=0}^{+\infty}\frac{M}{\alpha}\parallel {e}^{k}\parallel \right).$$In addition, we have$$\sum _{k=0}^{+\infty}{\lambda}_{k}\parallel C{x}_{B}^{k}C{x}^{*}{\parallel}^{2}\le \frac{1}{\gamma (2\beta \frac{\gamma}{\epsilon})}{\parallel {z}^{\circ}{z}^{*}\parallel}^{2}+S,$$$$S=\frac{M}{\alpha \gamma (2\beta \frac{\gamma}{\epsilon})}\sum _{k=0}^{+\infty}\parallel {e}^{k}\parallel +\frac{1}{\alpha {\beta}^{2}}\sum _{k=0}^{+\infty}\parallel {e}_{B}^{k}\parallel (\parallel {e}_{B}^{k}\parallel +2\parallel {z}^{\circ}{z}^{*}\parallel ).$$
 (5)
 If ${\lambda}_{k}\ge \underline{\lambda}>0$, then there exists ${z}^{*}\in Fix\left(T\right)$ such that the iterative sequence $\left\{{x}_{B}^{k}\right\}$ converges weakly to ${J}_{\gamma B}{z}^{*}\in zer(A+B+C)$.
 (6)
 If ${\lambda}_{k}\ge \underline{\lambda}>0$, then there exists ${z}^{*}\in Fix\left(T\right)$ such that the iterative sequence $\left\{{x}_{A}^{k}\right\}$ converges weakly to ${J}_{\gamma B}{z}^{*}\in zer(A+B+C)$.
 (7)
 Let ${\lambda}_{k}\ge \underline{\lambda}>0$ and assume that there exists ${z}^{*}\in Fix\left(T\right)$. Suppose that one of the following conditions hold:
 (a)
 A is uniformly monotone on every nonempty bounded subset of $dom\phantom{\rule{0.166667em}{0ex}}A$;
 (b)
 B is uniformly monotone on every nonempty bounded subset of $dom\phantom{\rule{0.166667em}{0ex}}B$;
 (c)
 C is demiregular at every point $x\in zer(A+B+C)$.
 (1)
 $\left\{{z}^{k}\right\}$ converges weakly to a solution $x\in zer(A+C)$;
 (2)
 $\parallel C{z}^{k}Cx\parallel \to 0$ as $k\to +\infty $ for ${\lambda}_{k}\ge \underline{\lambda}>0$;
 (3)
 $\parallel {J}_{\gamma A}({z}^{k}\gamma C{z}^{k}){z}^{k}\parallel \to 0$ as $k\to +\infty $;
 (4)
 Let ${\lambda}_{k}\ge \underline{\lambda}>0$ and let ${z}^{*}\in zer(A+C)$. Suppose that one of the following conditions holds:
 (a)
 A is uniformly monotone on every nonempty bounded subset of $dom\phantom{\rule{0.166667em}{0ex}}A$;
 (b)
 C is demiregular at each point $x\in zer(A+C)$.
 (1)
 $\left\{{z}^{k}\right\}$ converges weakly to a fixed point of T;
 (2)
 $\parallel {J}_{\gamma A}(2{J}_{\gamma B}\left({z}^{k}\right){z}^{k}){J}_{\gamma B}\left({z}^{k}\right)\parallel \to 0$ as $k\to +\infty $;
 (3)
 Let ${\lambda}_{k}\ge \underline{\lambda}>0$ and ${z}^{*}$ be a fixed point of T. Then the iterative sequence $\left\{{x}_{B}^{k}\right\}$ converges weakly to ${J}_{\gamma B}{z}^{*}\in zer(A+B)$;
 (4)
 Let ${\lambda}_{k}\ge \underline{\lambda}>0$ and ${z}^{*}$ be a fixed point of T. Then the iterative sequence $\left\{{x}_{A}^{k}\right\}$ converges weakly to ${J}_{\gamma B}{z}^{*}\in zer(A+B)$;
 (5)
 Let ${\lambda}_{k}\ge \underline{\lambda}>0$ and let ${z}^{*}\in zer(A+B)$. Suppose that one of the following conditions holds:
 (a)
 A is uniformly monotone on every nonempty bounded subset of $dom\phantom{\rule{0.166667em}{0ex}}A$;
 (b)
 B is uniformly monotone on every nonempty bounded subset of $dom\phantom{\rule{0.166667em}{0ex}}B$.
4. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
 Goldstein, A.A. Convex programming in Hilbert space. Bull. Am. Math. Soc. 1964, 70, 709–710. [Google Scholar] [CrossRef]
 Lions, P.L.; Mercier, B. Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 1979, 16, 964–979. [Google Scholar] [CrossRef]
 Combettes, P.L.; Pesquet, J.C. A DouglasRachford splitting approach to nonsmooth convex variational signal recovery. IEEE J. Sel. Top. Signal Process. 2007, 1, 564–574. [Google Scholar] [CrossRef]
 Tseng, P. A modified forwardbackward splitting method for maximal monotone mappings. SIAM J. Control Optim. 2000, 38, 431–446. [Google Scholar] [CrossRef]
 Beck, A.; Teboulle, M. Fast gradientbased algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 2009, 18, 2419–2434. [Google Scholar] [CrossRef] [PubMed]
 Beck, A.; Teboulle, M. A fast iterative shrinkagethresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2009, 2, 183–202. [Google Scholar] [CrossRef]
 Loris, I.; Verhoeven, C. On a generalization of the iterative softthresholding algorithm for the case of nonseparable penalty. Inverse Probl. 2011, 27, 125007. [Google Scholar] [CrossRef] [Green Version]
 Chen, P.J.; Huang, J.G.; Zhang, X.Q. A primaldual fixed point algorithm for convex separable minimization with applications to image restoration. Inverse Probl. 2013, 29, 025011. [Google Scholar] [CrossRef]
 Boyd, S.; Parikh, N.; Chu, E.; Peleato, B.; Eckstein, J. Distrituted optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 2010, 3, 1–122. [Google Scholar] [CrossRef]
 Yang, J.F.; Yuan, X.M. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Math. Comput. 2013, 82, 301–329. [Google Scholar] [CrossRef]
 He, B.S.; Liu, H.; Wang, Z.R.; Yuan, X.M. A strictly contractive PeacemanRachford splitting method for convex programming. SIAM J. Optim. 2014, 24, 1011–1040. [Google Scholar] [CrossRef] [PubMed]
 Chambolle, A.; Pock, T. A firstorder primaldual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 2011, 40, 120–145. [Google Scholar] [CrossRef]
 He, B.S.; Yuan, X.M. Convergence analysis of primaldual algorithms for a saddlepoint problem: From contraction perspective. SIAM J. Imaging Sci. 2012, 5, 119–149. [Google Scholar] [CrossRef]
 Condat, L. A primaldual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 2013, 158, 460–479. [Google Scholar] [CrossRef] [Green Version]
 Vũ, B.C. A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 2013, 38, 667–681. [Google Scholar] [CrossRef]
 Yu, Y.C.; Peng, J.G. A modified primaldual method with applications to some sparse recovery problems. Appl. Math. Comput. 2018, 333, 76–94. [Google Scholar] [CrossRef]
 Combettes, P.L.; Pesquet, J.C. Proximal splitting methods in signal processing. In FixedPoint Algorithm for Inverse Problems in Science and Engineering; Bauschke, H.H., Burachik, R., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H., Eds.; Springer: New York, NY, USA, 2010; pp. 185–212. [Google Scholar]
 Komodakis, N.; Pesquet, J.C. Playing with duality: An overview of recent primaldual approaches for solving largescale optimization problems. IEEE Signal Process. Mag. 2015, 32, 31–54. [Google Scholar] [CrossRef]
 Parikh, N.; Boyd, S. Proximal algorithms. Found. Trends Optim. 2014, 1, 123–231. [Google Scholar] [CrossRef]
 Raguet, H.; Fadili, J.; Peyré, G. A generalized forwardbackward splitting. SIAM J. Imaging Sci. 2013, 6, 1199–1226. [Google Scholar] [CrossRef]
 Raguet, H.; Landrieu, L. Preconditioning of a generalized forwardbackward splitting and application to optimization on graphs. SIAM J. Imaging Sci. 2015, 8, 2706–2739. [Google Scholar] [CrossRef]
 Combettes, P.L.; Vũ, B.C. Variable metric forwardbackward splitting with applications to monotone inclusions in duality. Optimization 2014, 63, 1289–1318. [Google Scholar] [CrossRef]
 Latafat, P.; Patrinos, P. Asymmetric forwardbackwardadjoint splitting for solving monotone inclusions involving three operators. Comput. Optim. Appl. 2017, 68, 57–93. [Google Scholar] [CrossRef]
 Lorenz, D.A.; Pock, T. An inertial forwardbackward algorithm for monotone inclusions. J. Math. Imaging Vis. 2015, 51, 311–325. [Google Scholar] [CrossRef]
 Davis, D.; Yin, W.T. A threeoperator splitting scheme and its optimization applications. SetValued Var. Anal. 2017, 25, 829–858. [Google Scholar] [CrossRef]
 Combettes, P.L.; Wajs, V.R. Signal recovery by proximal forwardbackward splitting. Multiscale Model. Simul. 2005, 4, 1168–1200. [Google Scholar] [CrossRef]
 Eckstein, J.; Bertsekas, D. On the DouglasRachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 1992, 55, 293–318. [Google Scholar] [CrossRef]
 BriceñoArias, L.M. ForwardDouglasRachford splitting and forwardpartial inverse method for solving monotone inclusions. Optimization 2015, 64, 1239–1261. [Google Scholar] [CrossRef]
 Marin, M. Weak solutions in elasticity of dipolar porous materials. Math. Probl. Eng. 2008, 2008, 158908. [Google Scholar] [CrossRef]
 Sidky, E.Y.; Jørgensen, J.H.; Pan, X.C. Convex optimization problem prototyping for image reconstruction in computed tomography with the ChambollePock algorithm. Phys. Med. Biol. 2012, 57, 3065–3091. [Google Scholar] [CrossRef] [PubMed]
 Chirilǒ, A.; Marin, M. The theory of generalized thermoelasticity with fractional order strain for dipolar materials with double porosity. J. Mater. Sci. 2018, 53, 3470–3482. [Google Scholar] [CrossRef]
 Cevher, V.; Vũ, B.C.; Yurtsever, A. Stochastic ForwardDouglasRachford Splitting for Monotone Inclusions. 2018. Available online: https://infoscience.epfl.ch/record/215759/files/CVY2016_preprint.pdf (accessed on 25 August 2018).
 Yurtsever, A.; Vũ, B.C.; Cevher, V. Stochastic threecomposite convex minimization. In Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain, 5–10 December 2016; pp. 4322–4330. [Google Scholar]
 Pedregosa, F.; Gidel, G. Adaptive three operator splitting. In Proceedings of the 35th International Conference on Machine Learning, Stockholmsmässan, Stockholm, Sweden, 10–15 July 2018; pp. 4085–4094. [Google Scholar]
 Rockafellar, R.T. Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 1976, 14, 877–898. [Google Scholar] [CrossRef]
 Combettes, P.L. Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization 2004, 53, 475–504. [Google Scholar] [CrossRef]
 He, B.S.; Yuan, X.M. An accelerated inexact proximal point algorithm for convex minimization. J. Optim. Theory Appl. 2012, 154, 536–548. [Google Scholar] [CrossRef]
 Salzo, S.; Villa, S. Inexact and accelerated proximal point algorithm. J. Convex Anal. 2012, 19, 1167–1192. [Google Scholar]
 Villa, S.; Salzo, S.; Baldassarre, L.; Verri, A. Accelerated and inexact forwardbackward algorithms. SIAM J. Optim. 2013, 23, 1607–1633. [Google Scholar] [CrossRef]
 Schmidt, M.; Roux, N.L.; Bach, F. Convergence rates of inexact proximalgradient methods for convex optimization. In Proceedings of the Advances in Neural Information Processing Systems, Granada, Spain, 12–17 December 2011; pp. 1458–1466. [Google Scholar]
 Solodov, M.V.; Svaiter, B.F. An inexact hybrid generalized proximal point algorithm and some new results on the theory of Bregman functions. Math. Oper. Res. 2000, 25, 214–230. [Google Scholar] [CrossRef]
 Eckstein, J.; Yao, W. Relativeerror approximate versions of DouglasRachford splitting and special cases of the ADMM. Math. Program. 2018, 170, 417–444. [Google Scholar] [CrossRef]
 Alves, M.M.; Geremia, M. Iteration complexity of an inexact DouglasRachford method and of a DouglasRachfordTseng’s FB fouroperator splitting method for solving monotone inclusions. arXiv, 2017; arXiv:1711.11551. [Google Scholar]
 Solodov, M.V.; Svaiter, B.F. A unified framework for some inexact proximal point algorithms. Numer. Funct. Anal. Optim. 2001, 22, 1013–1035. [Google Scholar] [CrossRef]
 Iusem, A.N.; Pennanen, T.; Svaiter, B.F. Inexact variants of the proximal point algorithm without monotonicity. SIAM J. Optim. 2003, 13, 1080–1097. [Google Scholar] [CrossRef]
 Han, D. Inexact operator splitting methods with selfadaptive strategy for variational inequality problems. J. Optim. Theory Appl. 2007, 132, 227–243. [Google Scholar] [CrossRef]
 Parente, L.A.; Lotito, P.A.; Solodov, M.V. A class of inexact variable metric proximal point algorithms. SIAM J. Optim. 2008, 19, 240–260. [Google Scholar] [CrossRef]
 Chouzenoux, E.; Pesquet, J.C.; Repetti, A. Variable metric forwardbackward algorithm for minimizing the sum of a differentiable function and a convex function. J. Optim. Theory Appl. 2014, 162, 107–132. [Google Scholar] [CrossRef] [Green Version]
 Chancelier, J.P. Auxiliary problem principle and inexact variable metric forwardbackward algorithm for minimizing the sum of a differentiable function and a convex function. arXiv, 2015; arXiv:1508.02994. [Google Scholar]
 Kang, M.; Kang, M.; Jung, M. Inexact accelerated augmented Lagrangian methods. Comput. Optim. Appl. 2015, 62, 373–404. [Google Scholar] [CrossRef]
 Li, J.Y.; Wu, Z.Y.; Wu, C.Z.; Long, Q.; Wang, X.Y. An inexact dual fast gradientprojection method for separable convex optimization with linear coupled constraints. J. Optim. Theory Appl. 2016, 168, 153–171. [Google Scholar] [CrossRef]
 Reem, D.; De Pierro, A. A new convergence analysis and perturbation resilience of some accelerated proximal forwardbackward algorithms with errors. Inverse Probl. 2017, 33, 044001. [Google Scholar] [CrossRef]
 Bauschke, H.H.; Combettes, P.L. Convex Analysis and Monotone Operator Theory in Hilbert Spaces; Springer: New York, NY, USA, 2011. [Google Scholar]
 Byrne, C. A unified treatment of some iterative algorithms in signal processing and image reconstruction. Inverse Probl. 2004, 20, 103–120. [Google Scholar] [CrossRef]
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zong, C.; Tang, Y.; Cho, Y.J. Convergence Analysis of an Inexact ThreeOperator Splitting Algorithm. Symmetry 2018, 10, 563. https://doi.org/10.3390/sym10110563
Zong C, Tang Y, Cho YJ. Convergence Analysis of an Inexact ThreeOperator Splitting Algorithm. Symmetry. 2018; 10(11):563. https://doi.org/10.3390/sym10110563
Chicago/Turabian StyleZong, Chunxiang, Yuchao Tang, and Yeol Je Cho. 2018. "Convergence Analysis of an Inexact ThreeOperator Splitting Algorithm" Symmetry 10, no. 11: 563. https://doi.org/10.3390/sym10110563